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RANDOM VERSUS ORDERED SEQUENCING IN 
COMPUTER-ASSISTED INSTRUCTION 



Introducti on 



The 
has been 
tages of 



ordered or " glcaV sequencing of learning materials 
most frequently mentioned as one of the primary advan- 
programed Instructinno Recently a number of papers 
have appeared which present inconclusive and frequently contra- 
dictory results concerning the advantages or disadvantages of 
ordered sequences of Items In a program compared to random se- 
quences of Itemso Some of these studies report superior learn- 
ing for carefully organized instructional programs, while others 
report no differences between random and ordered presentation 
condltlonSo In the writer's judgment, this basic conflict must 
be resolved before research can proceed to the Investigation of 
more subtle sequencing and organizational variables in programed 
instruct! on o 



The present project examined the question of random versus 
ordered sequencing within the context of several other variables 
thought to be Important In determining the effects of program 
organi zat1 on o Few previous studies on this problem employed 
factorial designs to study the effects of Important Interactions 
between sequencing variables and other relevant varlableSo It 
is the investigators' general position that the benefits of careful 
sequences of learning materials will depend on the kinds of students 
being taught, the content of the instructional program, the kinds 
of educational outcomes being sought (for example, recall of 
facts as compared to the learning of principles and transfer to 
new situations), and the care and extent to which an experimental 
version of an Instructional program is Initially sequenced. A 
number of recent single-factor experiments have given the im- 
pression that carefully organized learning sequences in instruc- 
tional programs "make no differenceo" The present study aimed 
at the more accurate statement that carefully organized learning 
sequences "make no difference," only under ce rt ai n conditions , 
and that course orge.n1 zati on 1s a highly critical variable in 
most educationally relevant situationso The relevant studies 
and rationale are developed more fully belowo 



The question of the importance of careful sequencing or 
conceptual organization of instructional programs is particularly 
important in the context of computer-assisted instruction which 
provides a high degree of flexibility in the planning, sequencing, 
and organization of course materials. To maximize the potential 
of computer-assisted instructional systems, unambiguous answers 
to questions concerning course sequencing must be found. 



2 



Rel ated Literature and Rati onalj. 

There has been -"uch. recent exploratory resea^ 

the instructional potential “J^^’^^a'^State University and Florida 
Current projects at The Pennsylvania „structi onal 

ir.s Uvefsitj ('w*) s." t!“j . 

system developed by I.B.M. testina college courses 

Watson Research Center ^®^f.®P’ufgrspLd*electronic computer, 

for presentation to students via a hign speeu « i.b.M. 

This^work stems from the ?Ltruction are exempli- 

Other approaches to (’964) , Breunfeld (1964), 

fied by tP® 1964 . Although these 

Stolurow and Davis ^ i nstruct vary m some 
different systems of computer assi flexibility of course 

respects they all have in commo g of adapting 

organization and “"learners^ Unfortunately, there 

is as yet little consistent ®''J,'^®"®® supeH to 

;s;5sa;i5 

f*Dr highly complex tasks. 

B. F. Skinner (1958). and R. ^1®®®/ <1961) ^ 

first to elucidate the principle Recently, proponents 

quenced items in j°?hat a wide variety of appro- 

of branching programs *^^^e argued typical branching 

program, learning ^®^li?g^!^anching pro^ provides individualized 
own responses* thus, the ^ number of studies have 

instructional sequences. 3 „uied item sequences in auto- 

appeared in which random or or 

instructional programs ”rase^ and Roe (1962) compared 

••logical" item sequences Roe, students on a program 
the performance of 36 freshman psyc „ere assigned 

in elementary ^ “^ILr half to an “ordered" sequence, 

to a scrambled sequence, the ^ criterion test 

Nonsignificant differences were o^^ined in the^cr^^^^^ 

performance of the two < ?p'^einpriment however, seem to indicate 
results of this small scale e p y.eouire the careful sequencing 

that college level students m y ^^ysiy ^gen supposed.' (Roe, 

of autoinstructional items as haa previuub.jr 

Case, & Roe, 1962 , p. 104). 

u.t, .,d «;‘?!; 

of second grade children a . differences were 

;"ounr\rerlS^iiors^o?ri:drd"Uri n was no evidence that 
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item scrambling Impaired learning. The studies 

important point which has frequently been ignored ^ * 
The posttest performance Indicated that the ordered 
failed to teach the material very effectivelyo That two poorly 
oraani zed oroqranis produce equivalent posttest achievenient is 
nSt surprising StSdies of sequencing variables must provide some 
indication of the sequential characteristics of the ordered 
program. 

Hamilton (1964) obtained a complex interaction , .v 

random versus ordered sequencing and erl 

versus nonspecific (read frames containing the correct answer} 
rlspSnd?^g These data indicated that the random-nonspecific 

combination was the most successful of ^"Jet" 

tunately, however, these data are very difficult 

since the specific versus nonspecific info^. 

confounded with the effects of information f®®'*^®®*'- 

mation feedback was given in the specific response condition. 

In contrast to results which question the '•"'P?'^*?"®®.®^. 
carefully sequenced instructional materials, are st®®’®® „ 

have obtained significant differences associated with sequencing 
of material in complex verbal learning. II 1ustra,_ive studies 
are those of Gagnd^ (1962) and Evans (I960). Gagne presented a 
procedure for organizing tasks in terms of a knowledge hierarchy. 
According to this point of view complex tasks may be broken down 
into compoment prerequisite tasks which must be learned before 
su«essT?e steps in the task may be mastered. Gagn^ provided an 
illustration of his procedure for seven subjects on a number 
series task. The procedure involves 

reouirements of the final task to prerequisite subtasks, inis 
procedure provides a useful method for constructing and analyzing 
self-instructional programs. Evans (1960) compared programs 
constructed by the “Ruleg" system to programs constructed by less 

srstema?1c methods. The Ruleg system P-^^d®®®?,* ^.f^rwas'^e! 
learning as the less systematic program, but less time was re 

quired to reach the obtained level. 

The following comments can be made concerning the unresolved 
issue of sequencing in programed instruction: 

rograms used in a number of 
in subject matter content. If 
knowledge of terms and there are 
items then careful sequencing is 
r such materials. Unrelated facts 
if presented in random order as 
the other hand, performance on a 
between concepts, and the under- 
ciples will be detrimentally 



1„ An examination of the p 
studies reveals much variability 
a program teaches vocabulary or 
few interrelationships among the 
not likely to be as important fo 
will probably be learned as well 
in some specified sequence. On 
program involving rel ati onships 
standing and application of prin 
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effected by random or scrambled presentati on o To provide data on 
this hypothesis, the present study employed two programs which 
have been developed as part of Penn State's project in computer- 
assisted instruction (OE-4-16-01 0) . These programs differed 
in the extent to which they taught principles and relationships 
as opposed to relatively unrelated factSo 

2o The writers agree with Lumsdaine (1963) who has commented 
on the importance (in studying sequencing) of the susceptibility 
of stimulus materials to the utilization of verbal mediating 
responseSo Some students undoubtedly provide their own conceptual 
organization to a scrambled program by 1 i nki ng vari ed parts of a 
randomized sequence of items with a verbal mediatoro Since 
students vary considerably in their verbal mediational ability 
it is likely that an interaction exists between random versus 
ordered sequencing and level of student verbal ability.. The 
effects of random sequencing are expected to be particularly 
detrimental to the learning of students exhibiting low verbal 
ability who would be unable to supply their own conceptual orga- 
nization of the subject matter^ A 2 x 2 factorial design was 
used to examine this interaction. The use of student verbal 
ability as one measured independent variable in the design also 
provided some indication of the sensitivity of the experiment 
as a whole since verbal ability is widely known to correlate 
with student achievement. The latter methodological suggestion 
has been made by Lumsdaine (1963). If one builds a strong 
variable into a research design and no significant effect is 
obtained for this variable, this would lead to the conclusion 
that the experiment was too insensitive to pick up differences 
due to other treatments. 



3o The literature on sequencing of instructional programs 
also suggests that variations in the objectives of learning or 
student outcomes as measured by criterion tests is an important 
variable. Most of the criterion tests employed in previous 
studies (insofar as can be determined from the published reports; 
measured the student's recall of facts learned 1n the program. 

If this is all that is required, then scrambling or randomizing 
item sequence may not be expected to have a serious detrimental 
effect on learning. On the other hand, if one were to measure 
student achievement of a number of other learning objectives, 

principle or relationship, or 
a principle in new problems, 
content might produce superior 
In the present study two criterion tests were 
testing recall of factual material presented 



such as, the understanding of a 
the student's ability to employ 
organized sequencing of program 
performance 
employed, one 



in the program, and the other testing the student's understanding 
of the principles involved and his ability to apply these princi- 
ples to problems not contained in the original instructional 
program (6.g»» a transfer task). 
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4. Finally, the investigators were convinced that the lack 
of a pretest was a methodological short coming in some earlier 
studies. The assumption has sometimes been made that students 
have “zero" knowledge about some content areas used in studies of 
programed instruction, and on this basis the pretest has oeen 
omitted. That the assumption of "zero" knowledge is highly 
untenable for adult or college populations has been demonstrated 
by Mager and Clark (1963). The present study employed a pre- 
posttest design to control the level of pretest performance. 



Ob.iecti ves 

Although the objectives of the study have already been 
discussed to a considerable extent above, they may be briefly 
summarized as follows;: 

a. To determine under what conditions careful 
sequencing of instructional programs "make 

a difference" in student learning within the 
context of computer-assisted instruction. 

Following appropriate null hypothesis tests, 
it was expected that scrambled item sequencing 
would have a detrimental effect on the learning 
of program content in which the mastery of some 
concepts and principles were prerequisite to 
the mastery of other concepts, (e.g., a program 
containing a conceptual hierarchy). 

b. To determine whether the effects of item se- 
quencing depended in part upon the instructional 
outcomes desired. If the objective of instruc- 
tion is recall of unrelated facts, item sequencing 
may be less important as a critical programing 
variable. When understanding of principles and 
transfer to new problems are desired, the effects 
of item sequencing may be more pronounced. 

c. To determine whether random as compared to 
ordered item sequences have a diffsrential effect 
on students of high as compared to low verbal 
ability. Randomized sequences were e:;.pected to 
have a more detrimental effect on the learning 

of low verbal ability students due to the in- 
ability of these students to reorganize the 
scrambled program on their own. 



o 

ERIC 
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Pilot Investigation 

This Is the first in a series of investigations of the 
effects of course sequencing in CAIo The primary purpose of 
the pilot experiment was to investigate the interaction between 
student aptitude and scrambled versus ordered seouencing of 
instruction. In contrast to earlier investigations, the present 
study employed a fairly lengthy instructional program of con- 
siderable difficulty for the average college studento The 
material used involved the learning of principles, mathematical 
problem solving, and contained a large number of sequential 
dependencies among the concepts taughto The specific objectives 
and predictions of the experiment were as follows: 

a) To determine under what conditions careful sequencing 

of instructional programs “make a difference" in student earning 
within the context of computer-assisted instruction. Following 
appropriate hypothesis tests, it was predicted that scrambled 
item sequencing would have a detrimental effect on student 
learning in a relatively lengthy, difficult program containing 
many sequential dependencies among concepts, e.g., when the 
mastery of some concepts and principles are prerequisite to the 
mastery of other concepts and principles. 

b) To determine whether scrambled as compared to ordered 

item sequences have a differential effect on students of high- 
as compared to low-verbal aptitude. An aptitude by sequencing 
interaction effect was predicted. Scrambled item sequences were 
expected to have a more detrimental effect on the , 

low verbal ability students than on the learning of high verbal 
ability students. It was expected that students of low-verbal 
ability would not have the conceptual skills required to re- 
organize the scrambled material. 



Methods and procedures 

Descr i pti on of i nstructi onal system ( CAI ) an^ cour^ materi aJ l.* 
The course uTeT”irrthe first experiment was a section of a irodern 
mathematics course which was developed for ^AI by the staff of 
the Computer Assisted Instruction Laboratory of the Pennsylvania 
State University. 1 The material selected contained instruction 



^The writers would lik»s to thank Professor Alan Riedesel and 
Marilyn Suydam of the Penn State Computer Assisted Instruction 
Laboratory who developed the original version of the Modern 
Mathematics program. 
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on the use of number systems with bases other than teno This 
learning task offered the advantage of being relatively difficult 
for college students to learn, and the material was unfamiliar to 
most students. The ordered version of the program presented 
subsets of Items in the following sequence: review of the base 

ten system; the concept of place value; the application of the 
concept of place value in base eight, base two, and base twelve 
number systems; transformations from one base to another; 
addition and subtraction in number systems with bases other 
than ten; and multiplication and division in number systems with 
bases other than ten. Previous experience with these course 
materials Indicated that most undergraduate college students 
could complete instruction in approximately two and one-half to 
three hours with a mean error rate of about fifteen per cent. 

The course materials used in the present study were pre- 
pared for CAI by means of a special computer language known as 
Coursewriter (Maher, 1964) developed by loBoMo computer scientists 
at the Thomas J. Watson Research Center, Yorktown Heights, New 
York, Using the Coursewri ter language, a course was programed 
including questions, problems, correct answers. Incorrect answers, 
knowledge of results, and remedial branches, all of which were 
stored on magnetic discs to which the computer had selective 
access to any part with an access time of less than one second. 

The computer was programed to accumulate and store all student 
errors and response latencies, and these data were later re- 
trieved for the investigators by means of a special program 
called Student Records , The scrambled sequence version of the 
number systems program was established by rearranging the se- 
quence of frames according to a table of random numbers. The 
scrambled sequence was then entered and stored on the magnetic 
discs as a separate course, 

1 The central computer used in the study was an I,B,M, 7010- 

1410 system located at I.B.M.'s Thomas J. Watson Research Center, 
Yorktown Heights, New York, The course materials in the form of 
questions, problems, prompts, etc., were teleprocessed over long 
distance telephone lines to student terminals on The Pennsylvania 
State University campus. The course was presented to students 
via an I.B.M, 1050 student terminal which consisted of a modified 
two-way electric typewriter, and a random access slide projector 
and tape recorder (the slide projector and tape recorder were not 
used In the present study). Questions and problems were typed 
out to the student, who typed his responses at the terminal. The 
student relayed his responses to the central computer which 
evaluated the response, provided knowledge of results, and se- 
quenced the student to the next appropriate step in the course. 

Subjects and procedures . Fifty-one undergraduate students 
in an introductory educational psychology class at The Pennsylvania 
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State University served as the S^s in the i nvestigati on o S^s wi th 
absolutely no previous typing experience were not included in the 
study. Two ^s were eliminated because a modern mathematics pre- 
test indicated they had previous knowledge of number systems with 
bases other than ten. One other ^ was eliminated because his 
Scholastic Apti tude Test scores (SAT) were not available. These 
el imi nati ons brought the total number of Ss to 48. 

Subjects were then subdivided into high- and 1 ov/-apti tude 
groups on the basis of their scores on the verbal Schol as tic 
Apti tude Tes t (SAT). The mean of the high group was 612 and the 
mean of the low group v/as 435 (SAT employs standard scores based 
on a mean of 500 and a standard deviation of 100). The original 
plan of the pilot investigation was to assign Ss within each of 
the high and low aptitude groups at random to the scrambled or 
ordered instructional treatment conditions. Although approximately 
half of the S^s were assigned to treatments at random, the random 
assignment of a large number of S^s had to be altered due to a 
number of programing “bugs" which developed at the last minute in 
the scrambled sequence program. For this reason, a larger number 
of S^s who were scheduled for the early experimental sessions were 
run in the ordered sequence condition, and a larger number of S.s 
scheduled for the later experimental sessions were run in the 
scrambled sequence condition. The investigator carefully ex- 
amined the two groups of subjects and in spite of the nonrandom 
assignment of some of the S^s , could find no evidence of selective 
factors which could account for the results obtained in the study. 
However, to provide additional assurance of the reliability of 
the results, a second independent triplication of the study was 
conducted (see Experiment I). 

The Ss reported to the Computer Assisted Instruction labora- 
tory individually and were given Form A of a 23-item achieve- 
ment test as a pre-treatment examination to test their prior 
knowledge of number systems with bases other than ten. Initially, 
each S was given a warm-up to familiarize him with the student 
typewriter terminal. After a warm-up period of about fifteen 
to thirty minutes, the ^ was allowed to begin instruction on the 
number systems program. At the completion of the program, S_ 
was given Form B of the 23-item achievement test on number 
systems. The reliability of Form B of the criterion measure 
estimated by the Hoyt technique was found to be .93 in an earlier 
study (Mitzel and Wodtke, 1965). The test-retest reliability of 
the criterion measure was also found to be ,93 for a one-week 
interval between testings in the earlier investigation. Following 
the achievement posttest, all ^s completed a Student Reaction 
Inventory consisting of S eman ti c Pi fferen ti a1 type items ( Osgood 
et. al . , 1957) designed to measure the student's attitude towards 
CAI. 
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All Ss ware allied to complete the In^tructlonal^.aten 
at theur own ratOc ug.A return the following day 

“.rn.’5S'‘S« s.r.5”K m,-,, 

frustrated by the scrambled sequence programo 

As previously mentioned, two Ss "f ®l^|”svltems. 

their pretest scores ’ s$2Vachi^ed in 
The remaining S_s who were . _-o+oct Seventy per cent 

the average, one-half P°’"* p^® P +|,e pretest Indicating 

systems with bases other than ten, 

test scores, errors made in „er frame, an effi- 

complete the hv^takina the ratio of criterion test per- 

ciency score obtained by taking the ratio or students' 

formance to were analyzed by means of a two 

attitudes towards CAI. ^®ce design with unequal numbers 

by two factorial analysis of varanced^ 

“v:rsrs^!orapt 1 trdr;-throth^rorrc*ram^ versus ordered pro- 
gram sequence. 



Results 

A prel 1 ml nary analys 1 s 1 ndi cated , th at al though ^the^high- 

and low-aptitude 9'"®“P?,*^ 1^^® sequence groups did not 
SAT measure, the scrambled and ordered !® 9 «ence^gro^p 

f ^^HH^ri^n’^a^anXsIs 0 ? qJaftUallve SAT scores produced 
nSnsfgnrficanrd 1 f?ere;ces aLg the four treatment groups 
employed in the studyo 

The distnbutions and "“'^[“^“.^j.^pe^heth^ the*«- 

dependent variables w®J'® a!!a'iv«;is of variance had been met* 
sumptions underlying the ^ deviate substantially from 

None of the distributions appeared t^ 

normalityo Hartley s wa-iance All of the F-ratios were 

assumption of homogeneity ^ F-ratio for the efficiency score 

nonsignificant except °'^®*/5®/ Sl level indicatif^g the 
was significant at less than :01 variable. In view 

presence of heterogeneity of ^ MQfiOl and Norton (1952) who 
bf the results obtained by Boneau (I960) and Norton i f 

found that heterogeneity of variance d^ "o? varlard for the 

the t-test or F-ratio, the heterog neity ^ results ob- 

pfficiencv score could not nave serious ly 

lalned in the present study. 
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Table 1 summarizes the results of the analyses of variance 
of three of the dependent variables, frequency of errors made in 
the program, per cent errors, and criterion test scoreo 
suits indicated that students in the scrambled sequence group 

made significantly more errors +u® students 

dents in the ordered sequence group (P <.0U1). 

In the scrambled sequence group were tendency 

remedial segments of the program (due to their ^ 

to make errors) than the students in the ordered group, the 
scrambled sequence group actually responded to 

lharthe ordered sequence group. The differences obtained in the 

total freauency of errors might have resulted from the fact that 

the students in the scrambled group simply responded . 

questions and thus had more opportunity to make 

ordered qroupo To control for this possibility, 

also computed based on per cent error scores. led^ 

1 this analysis also indicated that students in the scrambled 

kqSence griu^ made a significantly greater siS- 

than the ordered sequence group. In spi«.e of the ^9 

nificant sequencing main effect f?r frequency and percentage o^ 
errors, the sequencing main effect for the criterion iesc s^ore 
was nonsignificant. Considered together, *bese results indicated 
that although the scrambled sequence students made ®’ 90 ifican y 
more errors during instruction than the ordei^ed sequence |s. they 
apparently improved their performance during ^o?b'^oo-’oo oj®’ 
bv the end of the course, they performed approximately at the 

sLflevel as The ordered grotlp^n the ".^Sc! 

Hptaiied analvsis of the frequency of errors made during instruc 

tion was undertaken in Experiment I to determine whether students 
in the scrambled sequence group showed improvement from the 
beginning to the end of the course. 

The results reported in Table I also show an 
seauencinq interaction of borderline significance (P<.10) for 
th^fJeJIlIncy of errors and criterion test scores. However, 
the interactions which were obtained for these — 

not result from a decrement in the bSt"from 

aptitude qroup in the scrambled program as 
a decrement in the performance of the Hi gh apti tude Ss i n 
scrambled program. The results of the 

the tentative conclusion that scrambling the ^ tude 
program had little or no effect on the performance 

students, but produced a rather inter- 

formance of high-aptitude students, ^he graphs of tne i^er 
actions for the frequency of errors and critevion test variables 
are shown in Figs. 1 and 2. Both of these figures show the sharp 
drop in perform^ce of the high- aptitude students in the scram- 
bled sequence program. 



o 
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Table 1 

Analyses of Variance of Frequency of Errors, Per Cent Errors, 
and Criterion Scores for High- and Low-Aptitude Students 
in Scrambled and Ordered Sequence Conditions 



Source 


d.f. 


Frevquency 
of Errors 
F-ratios 


Per Cent 
Errors 
F-ratios 


Cri teri on 
Score 
F-ratios 


Apti tude 


1 


1.48 


2.42 


.27 


Sequencing 


1 


12.65*** 


1 1 .94** 


1.40 


Aptitude X Sequencing 


1 


3.96* 


.69 


3.62* 


Error 


44 


(529.26)® 


(801.60)® 


(32.07)^ 



^ Equals the mean square of the error term 
* P is less than JO 
** P is less than .01 
*** P is less than .001 
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Mean 1^-- 

criterion 13 
tes t 12 

performance 

11 




Low aptitude 
High aptitude 



ordered 



scrambled 

Program version 



Fig. 1. Criterion test performance of high- and low-aptitude 
croups taught by scrambled and ordered instructional programs. 

(N's equaled: HA-ordered = 17, HA-scrambled = 9, LA-ordered = 14, 

LA-scrambled = 8) 




Fig. 2, Mean number of errors of high- and low-aptitude students 
taught ^y scrambled and ordered programs. (N's equaled; HA- 
ordered =17, HA-scrambled = 9, LA-ordered =13, LA-scrambled = 9) 



The analyses of the other dependent variables are consistent 
with the results already reported. Table 2 shows the analyses 
of variance summaries for total instructional time, average 
response latency per frame, d the efficiency score measures. 
Students in the scrambled sequence group took significantly 
more time to complete the program than students in the ordered 
sequence group (P<.001). The scrambled sequence group took 
on the average 45 minutes longer to complete the program than 
the ordered sequence group. The longer total instructional time 
taken by the scrambled sequence group was not simply the result 
of their responding to more questions than the ordered group. 

The results reported in Table 2 also indicated that the scrambled 
sequence group on the average took longer to respond to individual 
frames of the program. The sequencing main effect for the average 
latency variable was significant at less than the .01 level. The 
scrambled sequence students took on the average 2.05 minutes per 
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frame to respond, while the students 1n the ordered sequence 
group averaged 1.59 minutes per response. 

The efficiency score reflects the amount learned per unit 

t, =h..h» " r,”” *;..- 

r«i‘ 

!r!E?!=?2. "i ^ 



Table 2 

Analyses of Variance of Total Instructional Time, Average 
Response Latency, and Efficiency Scores for High- 
and Low-Aptitude Students in Scrambled and 

^oniipnrp Conditions 









— 




Source 


dof 0 


Total 
Time 
F- rati os 


AvOo Respo 
Latency 
F- rati os 


Efficiency 
Score 
F- rati os 


Apti tude 


1 


.79 


U63 


2.07 


Sequencing 


1 


14.33*** 


10.77** 


7.50** 


Aptitude X Sequencing 


1 


2.52 


o02 


4.75* 


Error 


44 


(1591.03)® 


(.22)® 


(3475.43)® 



! ^ Equals the mean square of the error cerm 

I * P is less than o05 

** P is less than .01 
*** P is less than .001 

1 



j 



1 

J 



J 
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to be less efficient than the ordered program, even 

two groups eventually reach approximately comparable levels of 

criterion test performance. 

A statistically significant aptitude by sequencing^ 

ion was again obtained in the analysis of 

res (P <.05). This result is consistent with the luter 
res \i _ v»n* nn uariahlfiS™ The 



act 
score 
acti ons 



reported for the error and criterion 
scrambled instructional program P»'oduced the 
in the efficiency of performance 

c-i 7 uikirh shows the relatively sharp drop in the efficiency 
scores’of the high-aptitude group in the scrambled sequence 

condi ti on , 



Mean 

Effi ciency 
Score 




Fia 3 Mean efficiency scores of High- and low-aptitude 

students taught by scrambled and o^^^f^^^^PS^^Lnrdered = 
equaled: HA-ordered = 17, HA-scrambled - 9, LA-ordereo 

LA-scrambled = 8) 



13, 
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In considering the three interactions depicted in Figs. 1 
through 3, the reader should keep in mind that, in each case, 
the high-aptitude students in the scrambled seguence condition 
took the most instructional time but showed the £()_orest per- 
formance of all the treatment groups. 



We may summarize the results reported to this point as 
follows ; 

1) When considering the overall main effects for scrambled 
as compared to ordered sequencing conditions, the . 

dicated that scrambled presentation of course materials produced 
more errors in the program, increased total instructional time, 
increased response time per individual q^^estion, and . 

the efficiency of instruction as indicated by the amount learned 
pel Hit of time. If one examined only the overall effects of 
sequencing on criterion performance without taking into account 
the differences in instructional time, and differences in 5^“' 
dent aptitude, nonsignificant differences were 

for the sequencing main effects indicated that the students made 
more errors during instruction in the scrambled ^°lJr 

ditlon, but that they may have been able to compensate for the 
greater difficulty of the material by taking more time, by „ 

studying remedi al frames , and by reorganizing relevant information 
as it wL made available in the scrambled instructional sequence. 
Thus, by the end of instruction, the performance of students in 
the scrambled condition as measured by a criterion posttest was 
comparable to that of the students in the ordered condition. 



2) The effects on 
logically ordered instru 
on the ability of the le 
borderline significance) 
ordered course materials 
performance of hlgh-apti 
spent the longest time i 
for errors made in the p 
criterion measure (P*^ • 



a 
*■ n 



part 



student performance of scrambling 
ctional program seemed to depend 
arners. Three interactions (two of 
indicated that scrambling logically 
produced the greatest decrement in the 
tude students even though these students 
n instruction. This effect was obtained 
rogram (P <.10), performance . 

10), and the efficiency measure (P<-.05). 



Discussi on 

Considered in the context of several previous Investigations, 
the results of the pilot study suggest that the effects of scram- 



gicallv ordered instructional program depend on the 
the learning task and on individual differences in 



the 



bling a lo 

learners^ *^Several previous Investigations obtained no differences 
between Scrambled and "ordered" versions of 

qrams , when the programs were relatively short, and when each 
fl^aZI irthe prograS consisted of relatively discrete items of 
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Information. However, the results of P’’?* 
that when a relatively long, complex program ’"volving tne 
learning of principles and problem solving was ®";P’%®Jf 
scrambling increased within program errors and instructional 
time but did not affect the final criterion level of achievement. 

In the .iudgment of the writers, one of the most interesting 
tentative findings of the pilot study was that ’tern ®erambling 

impaired the performance of the students 

little effect on the performance of the low-aptitude 
iuSough lrap?itude By sequencing interaction had been P^icted 
in the study, scrambled item sequencing was expected 
mSst detrimLtal effect on the low-aptitude students. Jost the 
reverse occurred. This finding seems most interesting in view 
of the frequently heard maxim that "highly able students can 
learn by almost any method." 

What would account for the large decrements in the perfor- 
mance of the high-aptitude students in the scrambled 
condition assuming that the effect is ' 
low-aptitude students shown relatively little drop 

in the scrambled sequence condition? One nerformance by 

fnr the results obtained is that one cannot impair performance oy 

scrambling a program if performance i® aoti tudrstudIKts 

lame decrement in the performance of the hi gh-apti tuoe siuaeiii,» 

relultern each case from the fact that their 

quite high in the ordered condition and aStitSd^ 

students had father to d?op. The low-aptitude students, on the 
other hand started out in the middle range of ^cale m ^ 
ordered sequence condition. This explanation might P’®}'®’*’’® 
if there was evidence that the students were 
"floor" of the criterion test. However, the frequency di 
butions of performance on the criterion >"easure did not ’"^icate 

that positively skewed distributions °®®“;';;®^,”S’‘’5hrdist? - 
uihon floor effects are encountered. If anything, 

butions tended to be slightly negatively ® 
the criterion measure was not a "’“ItiPl®'?*'?’®?, > 
a po^tive chance score on the test was ^l^^es 

Ac^was already indicated, most students achieved pretest sco 
Sf zero In view of these characteristics of the >neasuring 
iLtrument, a mean score of 13 on the c*:’ "Si 
that considerable learning occurred ‘‘“If;'"? "nt, 

low-aptitude-orderod sequence group. In the w»^’Ters juagmeni. 




17 



A more probable explanation of the finding that item scram- 
bling produced larger decrements in the performance of the high- 
aptitude students than in the low-aptitude students was sug- 
gested by the results of the analysis of the student reaction 
data. The analyses of variance of the scales of the Student 
Reaction Inventory^ revealed significant effects for the three 
scales shown in Table 3. The interaction effect (again of border- 
line significance) for the Tense-Relaxed self-report rating scale 
was of special interest. The high-aptitude students in the 
scrambled sequence condition reported being more tense during 
instruction than each of the other three experimental groups. In 
addition, the high-aptitude students in the scrambled sequence 
condition reported the instruction to be "deeper" than the other 
three groupso This latter scale probably reflects the students* 
subjective perception of the difficulty of the programo Finally, 
the students in the scrambled sequence condition tended to rate 
the program as more inflexible than the students in the ordered 
condition. The students* self-reports on the Tense-Relaxed and 
Shallow-Deap scales suggest that the scrambled sequence program 
aroused the anxiety of the high-aptitude students. It is 
possible that the increased anxiety reported by the high-aptitude 
students in the scrambled sequence condition produced the decre- 
ment in their performance. 



Table 3 

Analyses of Variance of the Student Reaction Variables Tense- 
Relaxed, Shallow-Deep, Inflexible-Flexible for the High- 
and Low- Aptitude Students in the Scrambled and 
Ordered Sequence Conditions 



Source 


d.f. 


F- 

Tense- 

Relaxed 


-ratios 

Shallow- 

Deep 


Inflexible- 

Flexible 


Apti tude 


1 


.39 


1.65 


.04 


Sequencing 


1 


2.26 


.42 


5,38** 


AptituH?» X Sequencing 


1 


3.92* 


4.58** 


.002 


Error 


43 


(2.93)^ 


(1.31)® , 


(3.04)® 



^ Equals the mean square of the error term. 
* P is less than .10 
** P is less than .05 
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In view of the borderline P-values, the 
problems encountered in the pilot study, and the Jf^t that the 
direction of the differences were opposite to 
probably safest to withhold judgment on the nature 
tude by sequence interaction effect. Experiment I ,4 

below was designed to provide ^ methodologically improved repli- 
cation of the pilot experiment. 
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Scrambled versus Ordered Course Sequencing in 
Computer-Assisted Instruction: Experiment I 



Experiment I was undertaken to replicatei extend* and im- 
prove upon the pilot study. There were several problems encoun- 
tered in the pilot study: 



1 . 



2 . 



Only approximately one-half of the S_s were assigned to 
the experimental tref:tments at random. Due to a number 
of programing problems which developed at the l^st 
minute in the scrambled sequence program, the random 
assignment of a large number of Ss had to be altered. 
Although there did not appear to be any selection factors 
in assigning Ss to treatments which could have affected 
the results of the study, we cannot be totally certain 
that these effects were negligible. 

The modern mathematics CAI program employed in the pilot 
study contained remedial branching sequences. Thus, not 
all the Ss received identical instructional treatments. 
Those Ss^with high error rates were more likely to be 
sequenced through remedial material than S^s with low 
error rates. Although branching programs are most appro- 
priate for computer-assisted instruction, they present 
some obvious problems when one wants to make comparisons 
of experimental treatments which are unconfounded with 
the effects of remedial branching. The results of the 
pilot study suggested that the S^s in the scrambled 
sequence condition might have benefited 
number of remedial frames which they received. Although 
the scrambled sequence ^s made significantly more errors 
in the program, their performance at the ena of instruc- 
tion was not significantly different from that of the 
ordered sequence S^s as measured by an achievement post- 
test. 



3 The CAI instructional program employed in the pi'Jot study 
was in its preliminary stages of development. This 
oriQinal program had not been systematically evaluated 
or revised, and although student criterion test perfor- 
mance on the ordered version of the program was adequate 
for a first draft program (see Figure 1, page il / JT 
was decided that further experiments should be conducted 
on an improved version of the modern mathematics instruc- 
tional program. 



The major objectives of Experiment I 
same as those of the pilot investigation, 
methodological in an attempt to avoid the 



were essentially the 
The major changes were 
difficulties encountered 
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the preliminary experiment. The specific objectives were as 
follows: 

a) To determine the effects p"ig?am^wh^ ' 

’ ling a logically or^o^Sm irSied. and when 

relatively long and difficult program 

rLI?irn'=o/r-Sern''mlthemat1cs p 

^trLrt%"h^s^^rr1*t^Hr;r::rc^h^olln"a^ the /e^arnlng 

task for the j+udv was to examine a 

b) A second major P“j;P° 5 ® ultween several student Indivld- 

number of interactions between severa treatment 

ual difference «'eas«nes. and the exper.^^ interaction of 
(scrambled versus on'lened sequence^^ ,,bta1ned In the 
marginal statistical sign icanc . ^ 55 measured 

pilot study indicating that hi g p detrimentally 

by the Scholastic ^ Pt? tu . de igi|.w •.,,j auction than 

affecteiTTthi^ena-nb led sequent 

;;:%S,sr.S"S4" s i !; 

surorlsing, but tentative finding. 



Methods and Procedures 



Re vision of the cal 1 nstruct 1 o_nal E *-ogram 

^ _ +rk+aii\/ y*pvissd by Hflirold Ssnds 
The instrijctlonal program was ^(g.^pa-1 criterion achieve- 

of the Penn State ^AI Laboratory Th^ objLtives of Instruction, 
ment test (see Appendix \ revised in the hopes of producing 
The instructional bv cH test performance 

a more effective program as m^f ured by^cr;^^^ revised 

The following ^‘^^^®Hw°nf^worki nq with exponents and raising a 
program; 1) A review of working wi^cn^^ p^^ number system 

number to its power, 2 ) -i^cp value, 3 ) Convert a number 

and uif«derstand the p equivalent using an appropriate 

in any base to ^ts base ten ^ number from base ten to 

conversion algorithm, and 4 ) ^ conversion algorithmo 

any other base and revised programs are shown 

Proqram samples of the original - revised program 

in Appendix A. .^*'®^’?i,"?g^cuitable for the experiment, and that 
indicated that student learning as measured by 
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One major change In the in 
the elimination of the remedial 
development of the course as a 
this change was necessitated by 
imental treatments be identical 
of instruction. Since the bran 
to the performance of indi vidua 
to insure identical instruction 



structi.onal program consisted of 
branching segments and the 
linear program* As Indicated above 
the requirement that the exper- 
in all respects except the sequence 
ching program adapted automatically 
1 Ss, it was extremely difficult 
in the two experimental groups. 



The revised program consisted of 74 instructional 

ind took, on the ’ J”uT!l°sa|isf^ctory*in accomplishing 

‘hat the oroqram was reasonably sati siaci-ory + hi*;- 

T;.i I; 

legatively skewed about a mean of 17.03 ® deleted from 

total points on i°^d in th^^lo^ex^e^iment) . Thus. 

'^thoug^rLan number of Ss made 
: 4.u« tv nf the Ss aopear to have masterea 



jtively difficult task of converting numbers i 
>ases . 

The number systems course was programed 
... Jinn the coursewriter language Maher 1964) 

the Ss via CMI i^erision loo ic of the Coursewriter 

developed by loB.M. The . . Q^+hpr an ordered or scrambled 
language enabled us to provide either an oroerea or s 

sequence of frames from *5® branch statements following 

accomplished by the «se of conditional branch stocem 

®-5Lr"S""forscrtmbled®Sr "U" for unscrambled depending on the 
rxSHme^ntar?rra?^fnl $o°which he h^ 

At this time the computer stored a certain value in 

which identified the ^ _ the oroqram, the computer 

group. Following each problem in the progra^^ subject as 

®'>®‘:''?^*SUcr irprele^tld ?I?rSext ?rame in a pre-established 
an s subjGCt, / j*+aw*minprf from a tablG of random numbers). If 
random sequence a "u" sublet the computer presented 

the subject was identified as ^ ffo^-chart for one 

the next frame in expedient is shown in 

^^hirit^ateg^avoided the necessity of having to 
de?e\op iwo colpletHj Separate versions of the program. 



hhe writers wish to acknowledge 
Dr. David Gilman of the Penn State CAI 
this programing strategy. 



the contribution of 
Laboratory for suggesting 
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Criterion Test Score 



Fig. 4. Frequency distribution of the criterion test performance of 37 Ss in 
the ordered sequence. 
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IF STUDENT 
IS A "U“ 

■X- — I 

NEXT FRAME IN 
ORDERED SEQUENCE 



Fig. 5 



Flow-chart of an instructional frame 
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Description of the 



centra 

system located 
The reader w11^ 
course material 
Penn State camp 
New Yorko The 
entirely on the 
minals, located 
the 1nvest1gat1 
terminal, and s 
at the terminal 



y 



1 



the compytatlon 
recall that 1n the 
s were teleprocesse 
us from an IBM comp 
CAI system employed 
Penn State campuso 
in the Penn State 
opo The course was 
tudents entered the 



Experiment I was an IBM 1410 
center on the Penn State campus. 

pilot study, the experimental 
d to student terminals on the 
uter located 1n Yorktown Heights, 
in Experiment I was located 
Four IBM 1050 typewriter ter- 
CAI Laboratory were employed In 
typed out to the ^s at the 
1r responses by typing them in 



Subj ects and Procedures 

Eiqhty undergraduate students at the Pennsylvania State 
University served as the Ss in the experiment. Ss 
from a large section of introductory and 

a large section of a course in instructional media. The subject 
popciLion consisted predominantly of education majors. All Ss 
participated in the experiment during the fall term, 1966. Par 
ticipatlon in the experiment was strictly voluntary, and each 
S who completed all phases of the experiment received an hono- 
rarium of $4.00, The only requirement for participation in the 
study was that the student had not had previous course work in 
non-decimal number systems, and that his performance on the pre- 
test indicated little or no prior knowledge of the ®“hject matte* 
Ss were assigned at random to either the scrambled or order.d 

sequence condition. At the completion of ‘g,-- 

number of Ss for whom complete data were available on the ^ajor 
Sependent iariables of the study were 41 in the scramb ed se- 
quence condition and 34 In the ordered sequence condition. 

Ss reported to the CAI Laboratory individually and were met 
by a froctor. Four proctors were used in the study, one for 
each of the four student terminals. The following standard di- 
rections were read individually to each by a proctor. 



You 

0 

1 



A 

typewri 
typing 
shown h 
1 n an e 
by comp 
the CAI 
Is very 
on the 



In front of you a two-way typewriter commu- 
device used for computer-assisted instruction 
" presents problems to students over the 
ter, and the student enters h1s answer by 
it on the machineo In a moment you wi11 be 
ow to operate the typewriter. We are engaged 
xperlment to test the feasibility of teaching 
uter. In order to get a fair evaluation of 
(computer-assisted instruction) method, it 
important that you do the best job you can 
course which you will take. We also must 
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know how much knowledge you already 

matter to be taught. Have you ever had Math 200? 

Have you had any previous courses in *”9*? 
or college dealing with number syst®">s, 
other than ten? (If student says "yes explain that 
he has already had the material to be taught, that 
we can use him in a later experiment. excuse 
him from the experi.nent) . First, you willt® 9 ’ven 
a pretest to measure your achievement in the subaect. 
Following instruction you will be 9 iven two more 
tests One test will be taken immediately following 
instruction today, and the other will be taken tomorrow 
at a scheduled room. Your scheduled 

to take the test tomorrow is ,• 

student writes this down). Tomorrow you will also 
be given a short questionnaire to get 
to computer-assisted instruction. Remember in order 
to obtain the fee for participation in this experiment, 
you must complete the entire experiment. 

Before we begin, I want to get some general 
information from you. (Fill out file card on student). 

I want to see how you can do on this pretest, 
just to check on your knowledge this subject. 

I know that you probably have not 

on this topic, and that you may not be to do 

many of the problems. Don't worry about that. Just 

do the best you can. If you cannot do ^ 

just leave it blank. I will return in a few minutes 

to see how you are doing. 

Before we begin the actual instruction, the*;® 
are a few more things you may want to •'®°w. Although 
the time to complete the course varies depending 
upon the particular student, students on the average 
have taken about two hours to finish the course. 

After th0 first hour there will be a ._ 

5o take the break, and when it is time to start working 
again. There are two handouts you “ill need while 
working through the program. At several P°”’ts in 
thr«urse the typewriter will refer you to certain 
exhibits. These are the ®j|hihits you will ne .^ 

Rp cure to refer to the exhibits at the appropriate 
Uml in addition, to the exhibits, there will be 
times when the computer will ask y®® 

a problem using paper and P®"?’ ^ k^thr 
your solutions with the solution typed out by the 
computer. When you are asked to solve a problem 



erJc 
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using paper and pencil show all your work and the 
answer in the appropriate space on this work sheeto 
As you work through the problems you are not to look 
back at previous problems in the courseo 

If you suspect any difficulty with the machine, 
call on the proctor for assistance® However, the 
proctor has been instructed to give you no help with 
the problems or the content of the course® Do you 
have any questions before we begin? 

After the reading of the directions the S was given an opportunity 
to solve the problems on the pretest. Most S^s had no how to 

solve the pretest problems. Ninety per cent of the Ss obtained 
pretest scores of zero, and only two S^s obtained pretest scores 
greater than one. 

Following the pretest, ^s were given a brief explanation of 
how to operate the typewriter terminal, and the proctor assisted 
in signing the student on to the appropriate experimental version 
of the program® The proctor remained with the ^ through the first 
few frames of the program to insure that he had learned how to 
enter his responses at the terminal. The was then left to work 
through the program at his own rate. A five-minute rest period 
was included approximately half-way through the instruction® At 
the completion of instruction the S_s were administered an alter- 
nate form of the pretest to measure their ability to perform the 
number systems conversions. The following day, the S^s returned 
to the CAI Laboratory to take a transfer test and the Student. 
Reaction Inventory measure of attitude towards CAI® In a few 
cases a weekend or several days intervened between instruction 
and the administration of the transfer and attitude measures, 
however, in no instance was the interval between instruction and 
the transfer measure greater than 4 days® 



In summary, the independent 
consisted of one mani pul atable t 
ordered sequence of instruction) 
ference variables (verbal SAT, q 
grade point average), and sex. 
of the number of errors made in 
plete the program, scores on the 
test scores, and attitude toward 
Reaction I nventory . The latter 
Appendix!. As reported above, 
the achievement posttest was fou 
test was found to be exceedingly 
ability could not be adequately 
Brown (1966) obtained a Hoyt rel 
Reaction Inventory. 



variables of the investigation 
reatment effect (scrambled versus 
, four measured individual dif- 
uantitative SAT, total SAT, and 
The dependent variables consisted 
the program, time taken to corn- 
achievement posttest, transfer 
s CAI as measured by the Student 
three measures are reproduced in 
the test-retest reliability of 
nd to be .93® Since the transfer 
difficult for the S s , its reli- 
assessed in the present study, 
lability of .88 for the Student 
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Resul ts 



The results were analyzed by means of single-factor analyses 
of variance and within groups correlations. The analyses of 
variance provided information on the overall effects of the scram- 
bled versus ordered sequence on student learning and transfer. 

The within groups correlational analysis provi ded i nformati on on 
the interactions between the student individual difference measures 
and the experimental treatment. 



The main effect of program sequence on student learning 

Means, standard deviations, and the analyses of variance 
summaries are shown in Table 4 for the instructional time and 
error measures. These results are consistent with the results 
obtained in the pilot experiment. Scrambling the sequence of 
instruction increased instructional time (P "^.10) and Increased 
the number of er>^ors made in the program (P <.001). Hov^ever , 
when one examines the analyses of the achievement postte^t and 
trrnsfer measures shown in Table 5, one finds nonsignificant 
difrerences between the scrambled and ordered sequence conditions. 
This result is again consistent with the results of Experiment I 
(see Table 1, page 11). In the pilot experiment, the Ss in the 
scrambled sequence condition made more errors, and took longer 
to complete the program than the ^s in the ordered sequence 
condition, but by the end of instruction the scrambled sequence 
Ss were about on a par with the ordered sequence S_s as measured 
Fy the achievement posttest. In Experiment I, there were again 
differences between the treatment groups on errors and instruc- 
tional time, but no significant differences on the achievement 
posttest administered at the termination of instruction. These 
data support the conclusion that although the scrambled sequence 
is detrimental in terms of errors produced and instructional 
time, Ss are able to overcome the difficulties produced by the 
scrambled sequence and eventually arrive at a level of criterion 
performance comparable to the ordered sequence group. 

Further light is shed on this finding by examining the errors 
made per block of ten frames by the two treatment groups as shown 
in Figure 6, The ordered sequence group shows a gradual increase 
in the number of errors over the course of the program beginning 
at a mean of 1.2 errors persubject in the first ten frames, and 
rising to a mean of 4.1 in the last twelve frames. This increase 
in errors for the ordered group simply reflects the gradually 
incroasing difficulty of the problems in the programr While early 
frames involved relatively simple problems such as raising a 
number to its power, and expanding a number, the program builds in 
difficulty to criterion level problems involving the conversion of 
a number in one base, to its equivalent in another base. Problems 
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Ta|)le 4 



Summary Statistics and Analysis of Variance 
for the Time and Error Scores 





Scrambled 


’■ ■■ 

Ordered 


Analysis of Variance 




Dependent 
Vari able 


Sequence 

(n=43) 

X s.d. 


Sequence 

(n=^37) 

■ ! ' ' ■ - IN 

X ?,d. 


Mean Squares 
Treatment Within 
(1 d.f.) (78 d.f.) 


F 


P 


Total 
Instruc- 
tional Time 
(minutes) 


130.70 26.80 


121.00 20.40 


1877,80 


570; 70 


3^24 


<.10 


Errors 


16.30 5.68 


11.60 5.40 

-^■111 1 > ' 1 ' i ' 


422.13 


30.83 


13.69 


< .001 







Table 5 








Summary Statistics and Analysis of Variance 
for the Achievement Postte^t, Transfer Test, 
and Attitude Measures 




Dependent 

Variable 


Scrambled 

Sequence 

(n=41) 

X s.d. 


Ordered 

Seqqence 

(n=34) 

X s.d. 


1 

Analysis of Variance 
Mean^Squares 
Treatment witFin 

(1 d.f.) (73 d.f.) F P 


Achievement 

Posttest 


18.50 9.40 


17.10 5.00 


36.40 59.80 


< 1,0 


Transfer 


5.29 4,40 


4.76 4.56 


5.18 20.00 


< 1.0 


Attitude 
Towards CAI 


77.10 9.89 


74.60 12,40 


113.80 122.90 


<1.0 
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in the later stages of the program t-equire the 

int^nrate and aoDlv what he has learned in the early stages or ine 

nrnnrsm The error curve labeled "effects of scrambling on orig- 
?^al i?4m lequence"" sKows ?he effects of scrambling on the errors 

produced to frames as they were originally *the oer- 

ordered oroaranio This curve starts at the same point as the per 

formance^curve for the ordered sequence 

diverges to a considerably higher error rate in ® 

later^frames. The comparison of these two performance curves 
ISEporHThl vilw that%he original "ordered" Program did in fact 
contain seauentiallv dependent frames. Thus, when frames 31 
arp taken out of their original ordered sequence and scattered at 

random throughout the program frlm« 51-62 were 

to these frames increases from 1.5 to 3.5. When frames si were 

scrambled, the mean number of errors . later staoes'of 

The ability to solve problems in the middle and later stages or 

the roEra^; clearly depends to some e><tent on now e ge acqu red 
tn the parlier staoes of the program. When the original sequence 

was altered, the number of errors ioo;;ea®®2;e 6®ir?he one"'slEply 
interestinq performance curve shown in Figure b is tne one simp y 

”, sc.bl.d pr09..«, the »rr«rs SKa"';' 

ordered sequential dependency among the srlre 

problem called E. The depends upon an^understanding 

rr^he^learE E %’de e^dl Sjon ^ ^ndlrs^nd^ng of A and B. 
aEd so on. The learning task is now V“c°"andT anS tl 

dents in the naturally and A* The latter group 

another group in tne sequence D,C,» instruction. Questions 

of course will make more hf^ancwered correctly because the 

pertaining to concept D cannot be answered 

irirhighly Eroblble that’immediately after instruction on 
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IlTs' s’reU Ulollrl Irronce^tfc. and 

nor“ of Ihl relevant ilformati on becomes available but complete 

SnS^rstan^ing may. still be «rL°L5rre^J exposed to 

^iril^th^reufanrcrncep?^ ow^?ha^^^ ayai able. 

ciJceJt B mS^b^understeSd. and once concept B is available 

concept C may be «^,rea”"”Ris interpretation 

understood then concept Dbew J-giate the relevant informa- 

suggests tha a7ailable by the program to the previously 

tion as It IS made avaiiaoie oy ^ oresented out of sequence. 

poorly understood concepts whi ”®[|^Letati on would be that 

The major >^equirement of such an ^cts of previous 

the Ss were capable of recalling relevant a p information 

problems at some later additional requirement 

'needed to solve thim became a ailab^e 

would seem to be that the na oroaram to problems 

relevance of material provided y‘®e?I?iSrieans heavily 

"'’^tL^coanintrSceislrSf the Ss which allow them to integrate 
rnd"organrzr’irfo?:atfon%egard1esl of the sequence of presen- 
tati on » 

A 1 rt# thp measure of attitude towards CAI produced 

Analysis^of between the scrambled and ordered 

nonsignificant differences oeiv. appear to be any sex 

diffeiencerir performance on’the dependent variables and no sex 
by treatment interactions. 



Treatment by aptitude i ntereicti _on effe _ c t ^ 

Two methods were used to examine the interactions between 

?KU:S;g1i.”SK3 

lations of the ">easur8d ID var able with treatment. 

within each level of the ’iqn^ dif- 

nonsignificant. If tne «. -nin yi p aptitude by treat- 

cantly different ’"^’cating t p e.en piot'the regression 



32 



sijoaested bv Cronbach (1957) as a method for bridging the gap 
between the two traditional disciplines 

The approach to S^mr^nce^^rrrable^ 

*b;^rrrnSrch!"Lrbe“ront;:rt:d‘‘:4lh‘|lTo.ewha 

approach to examining such p[ljent variable 

study which involves both a measured experimental treat- 

such as test anxiety, aptitude, .tc , and an axpenme 

ment, the Ss may be divided at the median 

anxiety or high and low aptitud g p • analysis of 

analyzed in the *"®?i^’“SmLharS"re fa^^^ inter- 

action between the letter procedure is the 

experimental treatment., ^*up assessment of interaction 

more common of the two methods, t , be the more 

effects by means of “It^in groups correlati^^ ^ 

preferred procedure since it ®vo - aroups, and treats 

tinuous score distribution ’”t g u-jgj^ gj g correlational 
what is essentially a P^°threnti% range of the 

problem examining the ’nteraction over tne en variance 

Lasured independent variable. The factor relation- 

approach may be the P^f®"!^„^f5nLpendent variable and the 
ships exist between the “^“g’^P/p^riment. the writers 

decided to examine the ’"teract.ons be^e^n apn 

by the SAT and course sequencing by ^g g^d the 

the within groups \p^«w.a-i of the more common fac- 

aptitude distributions into high und low group 

The correlations between the SAT scores, 
average (GPA), and the dependent vanables^comp,^ ^ 

for each of the two trea m ?.stu +be SAT scores differ signifl- 

good many of the «7f.fll?lcrioSs and general magnitude of the 
cantly from zero and the d. recti ons an 

relationships a*^®.c9''®’®h®"*t^tude by sequencing interactions 
parisons for examining ^he aptitude by seq g„®^gdependent 
involve the comparison of ^^e correlations treat- 

variable (SAT score) with a dependent var^_^^ evident 

ment conditions. P9*'t®oo5)P4-he SAT verbal and the achievement 

if the correlation between the SAT verbal an^ different 

posttest in the t?'^ambled condition was |^9";;^,gbies in the 

from the correlation between correlation between the SAT 

ordered sequence condition. The correianui 



Table 6 

Correlations Among the Dependent and Independent 
Variables Within Each of the Two Treatment Groups 
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verbal and the achievement posttest in the scrambled sequence 
condition was .57 while the correlation between these same two 
variables in the ordered condition was .38. Each forre.ation 

was transformed via Fisher's e 

between the two was tested for statistical 

the procedure outlined by McNemar 0962. P® 9 e 139). f 

ence between the correlations of ;57 ?iL seouence 

railv ST ani f 1 cant . Each correlation in the scramblea sequence 

condition was compared with the ptirrof 

two variables in the ordered condition. None of the pairs or 
correlations was significantly different, i^^i^ating that the 
aptitude by sequencing interactions were nonsignificant as 
assessed by the correlational analysis. 

In addition to the statistical nonsignificance of 
ferpnces between the correlations reported in Table b, is 
important to note that the direetiog of t**® 
consistent with the interactions obtained ^ 

The reader will recall that in the pilot study the high aptitude 

Ss were more detrimentally affected by to^hav^been 

than the low aptitude Ss, In order for this finding have been 

rpniirated in Experiment I. the correlations between the SAT 

scores and the dependent variables would have to have 

in the ordered sequence condition than in the scrambled sequence 

condition. On the contrary, the ® 

where the largest differences occurred, the the 

largest in the scrambled sequence condition rather 
ordered condition. Thus, the correlations between the SAT 
verbal, quantitative, and total scores, and the achievement post- 
test were 57, ,74» and ,80 respectively in the scrambled se- 

Suence condition, but the ^ f ‘'a^r^SJ^rllpectTv^^^ 

scrambled sequence condition, and -.46 in *9®.°^^!'^®+ !|^t|”ally 
condition. These results, although 9 °®®’ 9 nificant statistically^ 

are consistent with the results obtained oi’+ue 

are consistent in direction with t'’! 0 '^’ 9 inal hypothesis of the 
present series of investigations. The writers original expec 

Sas that scrambling the sequence of .^udrand’oerfor 

increase the covariance between measures of aptitude and perfor 

mance during instruction. 



The aptitude by 
by means of the more 
approach, S_s in each 
SAT scores were avail 
high and low aptitude 
2x2 factorial analy 
variable. The effect 
sequence interaction 



sequencing interactions were also examined 
traditional factorial analysis of variance 
of the two sequence groups for whom Veroai 
able were dichotomized at the median into 
groups. Table 7 shows the results of a 
sis of variance with errors as the dependent 
of major interest here is the aptitude by 
which was statistically significant at less 
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Table 7 



Analysis of Variance of Errors for High and Low 
Verbal SAT Groups in the Scrambled and Ordered 

Sequence Conditions 



T 



Source of 
Van ati on 


d.f. 


M.S. 


F, 


P 


Sequence 


1 


581,2 


22.8 


<•001 


Apti tude 


1 


386.9 


15.2 


<.001 


Sequence x Aptitude 


1 


108.3 


4.26 


CJiS 


Error 


39 


25.4 
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9 

L_J 




Sequence 



Fig. 7. Mean number of errors of high and low aptitude Ss taught 
by scrambled and ordered programs (N's equaled: HA-ordered = 11, 
HA-scrambled = 10, LA-ordered = 11, LA-scrambled = 11) 
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than the <,05 level » This Interaction has been plotted in Figure 

7 which can be compared with the plot of the same interaction in 
the pilot study (see Figure 2 on page 12). The significant 
interaction shown in Figure 7 indicates that the scrambled se- 
quence had the most detrimental effect on the performance of the 
low aptitude Sso This finding is consistent with the original 
hypothesis of the study, but inconsistent with the findings of 
the pilot study where the scrambled instructional sequence 
appeared to have its most detrimental effect on the performance 
of high aptitude Ss. The discrepancy between the two studies 
does not appear to be explainable in terms of differences in the 
abolute levels of aptitude of the groups. In the pilot study the 
mean of the high verbal SAT group was 612 and that for the low 
group was 435, In Experiment I, the mean of the high group was 
612 and the mean of the low group was 477, 

The factorial analysis of variance with total instructional 
time as the dependent variable is shown in Table 8. The aptitude 
by sequence interaction was statistically significant at less 
than the .05 level. The plot of this interaction shown in Figure 

8 indicates that the scrambled sequence condition increased%_ 
Instructional time for the low aptitude S s , but produced a slight 
decrease in instructional time for the high aptitude £S. This 



Table 8 



Analysis of Variance of Total Instructional Time for 
High and Low Verbal SAT Groups in 
the Scrambled and Ordered 
Sequence Conditions 



Source of 
Variance 


dof. 


M.S. 


F 


P 


Sequence 


1 


597.17 


2.59 




Apti tude 


1 


3566.13 


15.45 


<.001 


Sequence X Aptitude 


1 


1178.11 


5.11 


<.05 


Error 


38 


230.74 







Low Aptitude 
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Fig. 8. Mean instructional time of high and low aptitude taught by scrambled and ordered 
programs (N's equaled: HA-ordered=ll , HA^Scrambled=10, LA-Ordered=ll , LA-$crambled-ll) 
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finding Is again consistent with the original hypothesis of the 
study, that the performance of low aptitude Ss would be more 
detrimentally affected by a scrambled sequence of instruction than 
the performance of high aptitude ^s, however, it is inconsistent 
with the results of the pilot study. 

Finally, the factorial analysis of variance with the achieve- 
ment posttest as the dependent variable is shown in Table 9o 
Here the aptitude by sequence interaction is statistically non- 
significant (F=2.70, P >.10) , however, again the direction of the 
differences is inconsistent with the results of the pilot study, 
but consistent with the other interactions obtained in Experi- 
ment lo The plot of the aptitude by sequence interaction shown 
in Figure 9 Indicates that the scrambled sequence produced a 
slight Increase in the posttest performance of the high aptitude 
Ss,'"but a slight decrease in the performance of the low aptitude 
Fs . The reader may wish to contrast the interaction plotted in 
Figure 9 with the interaction obtained for the same dependent 
variable in the pilot study as plotted in Figure 1 on page 11. 

The direction of the interaction obtained in Experiment I is again 
consistent with the original hypothesis that the sequence of 
Instruction interacts with S s ' aptitude such that a scrambled 
sequence is more detrimental to the performance of low aptitude 
Ss than high aptitude Ss. 



Table 9 



Analysis of Variance of Posttest Scores for 
High and Low Verbal SAT Groups 
in the Scrambled and Ordered 
Sequence Conditi ons 



Source of 
Vari ance 


d.f. 


M.S. 


F 


P 


Sequence 


1 


1.65 


< 1.0 




Apti tude 


1 


219.06 


9.90 


<•001 


Sequence x Aptitude 


1 


59.73 


2.70 




Error 


39 


22.12 
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Mean 

Posttest 

Scores 




Fio, 9. Mean Posttest Performance of High and Low aptitude Ss 
taught by Scrambled and Ordered Programed (N's equaled: HA-ordered = 11, 
HA-scrambled = 10, LA-ordered = 11, LA-scramb3ed * 11) 
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In summary, the analysis of the aptitude by sequencing 
interaction effects by the two methods of comparing within groups 
correlations, and the factorial analysis of variance produced 
results which although consistent in direction, were not consis- 
tent with regard to statistical significance. In general, the 
correlational analysis produced nonsignificant i 
while the factorial analysis of variance produced several 
actions which reached acceptable levels 

cance. These discrepancies between two methods of analyzing the 
same data raise some interesting questions concerning the 
appropriateness of these two procedures for assessing 
between individual difference measures and experimental treatments. 
For example, it would be particularly interesting to determine 
the relative power of these two statistical rnethods for detecting 
"true" interaction effects. In view of the 

problem of assessing interactions between individual difference 
variables and experimental treatments the two senior authors are 
presently conducting a methodological investigation of this 
probl emo 

In view of the inconsistency of the results of the pilot 
study and Experiment I concerning the nature of the 
between student aptitude and instructional sequence, one must 

exercise caution in reaching any 

phenomenon from these two experiments. The first 

study produced an unexpected and rather unusual 

a borderline level of statistical significance suggesting that 

the scrambled sequence of instruction was most detrimental to 

thi performance 2f high aptitude Ss. The second study produced 

twO significant aptitude by sequencing Interactions for errors 

and total instructional time with the direction of ths affect 

opposite to that of the pilot experiment but consistent wiui 

the original predictions of the study. In ho 

scrambled sequence of instruction was most detrimental to the 
rerf^rmand of the low aptitude Ss on errors and instructional 
time and in the same direction, but not significantly so for 
Achievement PDSttest. These discrepancies between the two 
studies may have resulted from the partially nonrandom assign- 
men? of T/u the pilot study (resulting n the 
unknown selection factor), or the change in the 
program resulting from the revision of the program prior to 

Experiment I. 
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Scrambled Versus Ordered Course Seqirncing in 
Computer-Assisted Instruction: Experiment II 



Previous Investigations produced inconsistent findings con- 
cerning the importance of ordered sequencing in programed instruc- 
tionc Evans (I960) and Stolurow (1964) reported superior learning 
in carefully organized instructional programs, while Roe, Case, 
and Roe (1962) and Levin and Baker (1963) reported nonsignificant 
differences between scrambled and ordered sequence conditions,, 

One experiment (Hamilton, 1964) reported differences in student 
learning favoring a scrambled sequence condition. Since these 
previous investigations employed different subject matters, one 
possible explanation for the inconsistent findings may lie in the 
nature of the different subject matters employed. Some subject 
matters may involve a natural hierarchy of concepts and a se- 
quential dependency among successive stages of the courseo Indeed, 
the number systems program which was used in Experiments I was 
specifically selected because it appeared to have such hierarchical 
characteristics, and the performance data reported in Figure 6 
bear out this conclusion. The frequency of errors increased when 
the original sequence of frames was altered. The expectations 
of the investigators was that scrambling such an inherently 
ordered subject matter would have a highly detrimental effect on 
student learning. Somewhat contrary to these expectations, the 
results of Experiments I and the pilot study indicated that 
although the scrambled sequence led to a greater number of 
errors made during instruction, and to an increase in instruc- 
tional time, by the end of instruction the S_s in the scrambled 
condition were apparently able to master the material since 
their posttest problem solving was comparable to that of the 
ordered sequence S^s, 

Not all subject matters contain the conceptual structure 
inherent in such fields as modern mathematics. Performance in 
subject matters consisting of a set of relatively discrete 
facts, such as the learning of vocabulary, knowledge of terms, 
and anatomy, would not be expected to be detrimentally affected 
by varying the sequence of presentation. Experiment II was 
undertaken to provide information on the effects of a scrambled 
sequence on the learning of such a subject matter. In this 
phase of the study we were primarily concerned with the influence 
of subject matter characteristics on the effect of course s - 
quencing. By extending Experiments I to another subject matter 
it was possible to test the general i zabi 1 i ty of the effects 
obtained with the modern mathematics program. In addition, by 
comparing the results of the three experiments we hoped to gain 
some insight into the critical characteristics of subject matters 
which might interact with the sequence in which the material is 
taught. 



Method 



Descript -ion of the CAI instructi onal program 

Part of a CAI program written by Professor Bruce Siegenthale 
and Mr. Jeffrey Katzer of the Penn State CAI Laboratory was 
adapted for the present experiment. A more complete descnp- 
tion of the course may be found in an earlier report by Mitzel 
(1966). The first section of the course entitled Speech Pathol- 
ogy and Audiology was adapted for the study. The course segment 
consisted of instruction on the anatomy of the ear. The stu- 
dent's task was to learn the anatomical names of the parts of 
the ear. (A sample segment of this program is shown in Appen- 
dix Bo) Unlike the number systems program employed in the 
first two studies, the anatomy of the ear program did not appear 
to involve any inherently logical sequence. On an a pri ori 
basis, it did not appear to make much difference whether the 
student learned one anatomical part before or after another 
anatomical parto The task appeared to consist of a set of 
relatively discrete associations, and closely resembled a 
paired-associates learning task in which the stimulus item con- 
sisted of a part of the ear, and the response item consisted of 
the name of the part. The anatomical names were extremely rare 
in everyday usage thus, we could be relatively certain that 
unless the ^s had completed previous course work in the area, 
they could not make the appropriate responses. (Preliminary 
research with this program supported this assumption.) 

The Instruction was presented to the Ss by means of an 
IBM 1410 CAI system which was identical to the system used in 
experiment I. The student-subject matter interface consisted 
of an IBM 1050 typewriter terminal through which questions 
were displayed to the student and through which the student 
could enter his responses. In addition to the portion of the 
course which was administered via the typewriter, S.s had avail- 
able at the terminal a plastic model of the ear and several 
handout sheets containing line drawings of the ear. At several 
points in the program, the students were asked to identify a 
certain anatomical part on the model of the ear, and on the 
handout sheet, and to write the name of the appropriate part 
on the handout sheet. 

Sixty-four undergraduate students in an introductory educa- 
tional psychology class and in a course on ifistructional med.a 
served as the experimental S^s . The S^s volunteered for 
experimeryt and received an honorarium of either $2.00, $3.00, 
or $4.00 depending upon the average length of time j'eqji'' j;®® 
complete the experiment by the treatment group to which they 
had been assigned. The Ss were assigned at random to one of 
six treatment conditions in a 2 x 3 factorial analysis of vari- 
ance design. One main effect consisted of the scrambled versus 
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ordered sequence of instructiono The second main effect con- 
sisted of the number of times that the student was cycled 
through the programo The basic anatomy of the ear 
contained 22^i nstructi onal frames. Preliminary tryouts of the 
prS5rr?ndicIted that a,ost students could 
of the anatomical names after only one trial 
qram. For this reason it was necessary to vary the number of 

trials through the program so that the “L^e^ilSels 

seouence could be observed over a number of practice levels. 

The ra?n Tf?ect for trials consisted of t»ree evelsi one. 
two, and three trials through the program. This design also 
permitted the examination of the interaction . 

seouencinq variable and the amount of practice. It seemed 
probable that the effects of the scrambled sequence might be 
offset by increasing practice. 

Previous work with the anatomy of the ear 
that s?udents who had not had previous ® 

could not produce the correct anatomical names. The invest! 
qators therefore decided to dispense with the pretest and simply 
screened out those students who had completed previous course 
wo^fin audiology or related areas. Three students were elimi- 
nated on this basis. The experimental Ss reported to the CAI 
Laboratory individually and were met by a P»'OCtor. ®J^‘ „ , 

riving at the Laboratory, each S was escorted to an experimental 
soundproofed room containing the typewriter terminal. The 
proctor then read the following instructions to the S. 

You see in front of vou a two-way typewriter used 
for domputer-assisted instruction. A computer presents 
problems via the typewriter, and the student enters Ms 
answers by typing them on the machine. We are engaged 
a^LperimPt to test the fe?t’bil ty of teach ng by 

computer. In order to get a fa’/. ®''®’ ’ IL Purse 
important that you do the best job you can on th 

which you will take. 

Following instruction, you will 
You will also be required to come back for another 
IZsVof thi stPy. This second session will. require 

Pout 15 minutes of your time; jj^erif the 

be paid unless you participate in both phases 

experi ment . 

Before we begin the actual ii?st*'“Ction , there 
are a few more things you may want 

time to complete the course vanes depending upon tne 
oarticular student. There are auxiliary materials 
yp will need while working through th® .P*'°9»;am. If 
yPr proctor has not specifically explained this to 
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you, please remind him to do so now . Be sure to 
refer to these materials at the appropriate time. 

As you work through the problems, you are not to look 
at previous problems in the courseo 

If you suspect any difficulty with the equipment, 
call on the proctor for assistance. However, the 
proctor has been instructed to give you no help with 
the problems or the content of the course. Do you 
have any questions before we begin? 

Following the instructions, the proctor assisted the i in signing 
on to the appropriate experimental version of the course; either 
ordered sequence-one, two, or three trials, or scrambled sequence 
one, two, or three trials. As was the case in Experiment I, 
the scrambled sequence was again determinGd from a table of 
random numbers. The programming strategy employed in Experi- 
ment I (see Figure 5) was again employed in Experiment II to 
generate the six program versions from one basic ordered sequence 
program. At the time of sign-on, the identified himself to 
the computer as a member of a particular treatment group, and 
the computer presented the appropriate experimental version of 
the program. In the two- and three-trial conditions of the 
scrambled sequence groups, the same scrambled sequence was 
repeated on the second and third trials rather than compiling a 
new scrambled sequence for each trial. 

After the £ had signed on to his appropriate treatment 
condition, the proctor remained with him through the first few 
frames of the program to insure that the understood how to 
operate the terminal. Once assured that the S_ had adequately 
mastered the operation of the typewriter terminal, the proctor 
left the experimental room, and the S completed the Instructional 
phase of the experiment. Immediately after the ^ had completed 
instruction, he was given a posttest (see Appendix C) to measure 
his ability to recall the names of the parts of the ear which 
were taught in the program. The Kuder-Ri chardson formula 20 
reliability of the anatomy of the ear posttest was found to be 
.86 based on the sample of Ss in the ordered sequence condition. 
When the S had completed the posttest, he proceeded to complete 
the Student Reaction I nventory measure of attitude towards CAI. 
Following this phase of the experiment, each S_ was scheduled to 
return to the CAI Laboratory one month later to take a retention 
test to measure delayed recall. At this time, the Student 
Reaction Inventc/y was re-administered to obtain data on the 
stability of attitudes toward CAI over time. 

In summary, the independent variables of Exper^nient II 
consisted of the course sequence (scrambled or ordered) and the 
number of practice trials through the program ("one, two, or 
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three cycles). The dependent variables consisted of errors 
made during instruction, instructional time, immediate recall 
as measured by the posttest, delayed recall as measured by the 
one-month retention test, and attitude towards CAI as measured 
by the Student Reacti on Inventory . 

One important addition to Experiment II was the inclusion 
of the one-month retention measureo The Inclusion of a reten- 
tion measure seemed most appropriate in view of the fact that 
the instructional program primarily involved the recall of 
anatomical labels. However; the retention measure was included 
for still another reasono Wodtke (1967) suggested that some 
instructional variations may have their primary effects on 
delayed recall rather than on immediate postinstruction per- 
formance. For example, one might predict that the effects of 
an inadequate instructional sequence could be overcome by a 
subject on an immediate posttest as a result of the S_'s own 
ability to reorganize and restructure the material, but that 
when the S was required to retain the material over a period 
of time, the scrambled sequence might interfere with recall. 

The fact that a scrambled sequence produced an increase in the 
number of errors during instruction in the two previous experi- 
ments, suggests that a larger number of incorrect responses 
might exist In the S's response repertoire at the time of 
measuring delayed recall® It is also possible that the logical 
structure of an ordered program aids delayed recall. The 
provision to include the delayed posttest in the present experi- 
ment provided some evidence on the effects of course sequence 
on retention over time. 



Resul ts 



In general the results of Experiment II supported the 
experimenters expectations that the sequence of instruction 
in the anatomy of the ear program would not be as Important a 
variable as in the modern mathematics program® The mean number 
of errors committed during instruction by the six treatment 
groups is shown in Table 10, and the analysis of variance of 
these data is shown in Table 11. All effects were nonsignifi- 
cant statistically, although there was a slight tendency for 
the scrambled sequence groups to make more errors than the 
ordered sequence groups® Essentially the same results were 
obtained when instructional time was analyzed as the dependent 
variable. The mean total Instructional time for the six treat- 
ment groups is shown -In Table 12 and the analysis of variance 
of these data is shown in Table 13. The only clearly signifi- 
cant effect was that for cycles which is simply the rpult of 
the additional time required by the subjects in the three cycle 
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Table 10 



Mean Number of Errors for the Scrambled and 
Ordered Sequence Groups Completing One, Two, or 
Three Cycles Through the Program 



Sequence 



Ordered 



Scrambled 




Table 11 



Analysis of Variance of Total Errors for Scrambled and 
Ordered Sequence Groups Completing Dne, Two, or 
Three Cycles Through the Program 



Source 


d.f. 


M.o. 


F 


P 


Sequence 


1 


47,56 


1,46 


> .10 


Cycles 


2 


58.55 


1.80 


> .10 


Sequence x Cycles 


2 


9.89 


<1,0 




Error 


54 


32.43 
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Table 12 



Mean Total Instructional Time in Minute for the 
Scrambled and Ordered Sequence Groups Completing One, Two, and 

Three Cycles Through the Program 

Sequence 



Ordered 



Scrambled 



Number 

of 

Cycles 



X=15 

s.d. 8.31 


(n=9) 


X=21 

s.d. 9.34 


(n=14) 


X=27 




X=22 


s.d. 8.86 


(n=9) 


s.d. 8.85 


(n=10) 


X=26 




X=36 


s.d. 8.70 


(n=11) 


1 S.d. 8.83 


(n=11) 



Table 13 



Analysis of Variance of Total Instructional Time, for 
Scrambled and Ordered Sequence Groups Completing One, Two, or 

Three Trails Through the Program 



Source of 
Vari ati on 


d.f. 


M.S. 


F 


P 


Sequence 


1 


854,350 


2.15 


> .10 


Cycles 


2 


3,331,950 


8.40 


< .001 


Sequence x Trials 


2 


1,147,585 


2.89 


< .10 


Error 


58 


396,517 







Note: The analysis in this table is based on seconds rather than minutes as 

in table 12. 
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condition to complete three cycles through the programo (The 
reader will recall that these effects were statistically sig- 
nificant in the modern mathematics program the P being less 
than .10 for instructional time, and less than .001 for errors*) 

In an attempt to provide some additional insights 
effects of scrambled sequence upon errors committed during the 
course of instruction, a more detailed analysis of the error 
data was undertaken. The original analyses reported above were 
based upon a randomized groups dasign (ioO., separate groups of 
Ss were assigned at random to each of the six treatment combi- 
nations)o If each S's error score is obtained separately for 
each cycle through the program, it is possible to analyze the 
error data for the three-cycle and two-cycle groups as a two- 
way factorial analysis of variance with repeated measurements 
on the cycles effect. Although the sample sizes of these analy- 
ses were somewhat smaller than in the original analysis (since 
they are based on only part of the original sample), the repeated 
measures analysis provided the added control for within subject 
varlanceo Table 14 and Figure 10 contain the results of the 
repeated measures analysis for the scrambled and ordered sequence 
groups (n = 10 per group) which were run in the three cycle 
condition. The sequencing main effect was nonsignificant sta- 
tictically, the main effect due to cycles was significant at 
less than the .01 level, and the cycles by sequence interaction 
was significant at less than the .01 level. The significant 
cycles main effect reflects the improvement in performance as 
a function of repeated practice with the programo The signifi- 
cant cycles by sequence interaction suggests that students 
during the first cycle through the program were detrimentally 
effected by the scrambled sequence, but that by the time they 
had completed the second cycle, the sequencing effect was no 
longer evident. This finding is reminiscent of the finding in 
Experiment I, that the Ss in the scrambled sequence condition 
were able to overcome an initial increment in errors and eventu- 
ally achieved a level of performance commensurate with that of 
the ordered sequence group. However, the evidence for an initial 
decrement in performance for the scrambled sequence group in 
Experiment II is much less consistent than was the case In 
Experiment I. For example, if one examines the repeated measures 
analysis for the subjects in the two-cycle condition shown in 
Table 15 and Figure 11 one now finds that the cycles by sequence 
interaction is nonsignificant. . The only statistically signifi- 
cant effect in this analysis was that for trial<=. 

Figures 12 and 13 show an even more detailed breakdown of 
performance reflected in terms of the mean number of errors per 
blocks of five instructional frames for each cycle through the 
program. The peaks in mean errors in frames 11-15 (block No, 3) 
in the ordered sequence condition appear to reflect an increase in 
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Table 14 



Repeated Measures Analysis of Variance of Errors Made 1n the 
Program for the Scrambled and Ordered Sequence groups 
Completing Three Cycles Through the Program* 



Source 


S.S. 


d.f. 


M,S. 


F 


P 


Between Ss 


Z76.40 


19 


14,55 






A~ Sequence 


29.40 


1 


29.40 


2.14 




^s Within Groups 


:.247.00 


18 


13,72 






Within is 


1415.33 


40 


35,38 






B Trials 


850.63 


2 


425,32 


35.95 


< .01 


AB 


138.70 


2 


69.35 


5.86 


< ,01 


BX Only Within 
Groups 


426.00 


36 


11,83 







*Note: In this an the other repeated measures analyses, some Ss were thrown out 

of some cells at random in order to achieve proportionality. 



o 

ERIC 




I 

J 



C=>!siJ 

o 

ERIC 
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Table 15 



ant S?5ered leS^ncf W '' 
® nolletinq-Two Cycles Through ttre'Prograra 



Source 







Between Ss 


189.00 


15 


12.60 




A Sequence 


.12 


1 


.12 


< 1.0 


Ss Within Groups 


188.88 


14 


13.49 




Within Ss 


479.00 


16 


29.93 




B Trials 


388.00 


1 


338.00 


34.34 


AB 


3.13 


1 


3.13 


< 1.0 


BX Ss Within 
Groups 


137.87 


14 


9.84 
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BLOCKS OF 5 FRAMES 

Fig. 12o Mean number of errovs blfacks of five frames for the scrambled and 
ordered groups completing two cycles through the program 



*Four Frames 
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Fig. 13, Mean number of error by blocks of five frames for the scrambled and 
ordered groups completing three cycles through the program. 



*Four Frames Here 
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the difficylty level of those frames o In each of the two figures, 
the scrambled sequence appears to have produced an Increase in 
errors during the first ten frames, however, this effect is 
primarily due to the fact that the more difficult frames ll-l 
In the ordered program were displaced in the scrambled program 
to the earlier segments 1-5 and 6-10, thus, accounting for the 
poorer performance in these segments for the scrambled sequence 
groupso The performance decrements of the scrambled sequence 
groups during the first ten frames do not appear to . 

result of any disturbance of a "logical ^ 

program, but rather resulted from a change in position of more 

fficult frames o 



3-5 results are taken by the writers to support the con- 
tention that performance on the anatomy program was not as 
dependent upon a logical sequence of instruction as was per- 
formance on the modern mathematics program. 
modern mathematics program produced highly 

In errors, while scrambling the anatomy program 9®”®‘ T 

ally nonsignificant differences on the 
one can say concerning the effects of scrambling 
program, is that there may be the suggestion of a slight ’"itial 
Increase in errors during the early frames, which quickly dissi- 
pates as the student proceeds through the Pj;®9*'®"’” . . 

this effect was confounded in the present study with the diffi 
culty of the frames in the early segment of the scrambled program 

Further evidence for the impotence of the 
1n the anatomy program comes from the analysis of * 

pesttest and one-month measures. These data were analyzed by 
Lans of a three-way factorial repeated measures ®"®’y®^® 
variance. The three factors in the analysis were the number 
of practice cycles through the program (one, 
scrambled or ordered sequence, and immediate 
Dosttest (the single repeated measure) o The 

deviations for the various treatment combinations are shown in 
Tlbirie Arcan be seen from Table 16, the means within each 
of the tables for the immediate and delayed posttests ®^® 

similar indicating a lack of an effect for ?®J“® ®®’i,e5 

the cycles by sequence Interaction. The means do show a marked, 

cSnsiHeft drop from the immediate to the J"**’ ' 

eating considerable loss due to forgetting. The analysis ot 
variance of these data shown in Table 17 simply bears out what 
has already been seen by examining the means Table 16. The 

only statistically significant effect was that Tor the immediate 
VGfsus dGlayed posttGSt which rcflGcts thG dGcrcniGnt in r 
over time. ^The^nonsignificant interaction *’®T”®®" J®2f®l?5idence 
immediate versus delayed POSttest indicates the lack o 
for differential retention over time as a function of one or the 
other of the treatments c 



two , or three) , 
versus delayed 
means and standard 



Table 16 






1 

1 



f I 




Means for the Immediate and Delayed Retention Tests for 
the Six Treatment Combinations 

Immediate Posttest Sequence 



Number 

of 

Cycles 



Ordered S crambled 



X=12.1 

s.d. 4.17 n=8 


1 

X=13.5 

s.d. 2.12 ti=8 


X=12.1 

s.d. 4.09 n=10 


X=13.1 

s.d. 4.01 n=10 


Si=15.6 

s.d. 3 '. 


> 

5(=14.2 

s.d. 4.88 



One-month Retention Test 



Number 

of 

Cycles 



Ordered Scrambled 



^-1 

X=6.2 

s.d, 2.28 n=8 


X=5.9 

s.d. 4.70 n=8 


X=6.8 

s.d, 3.84 n=10 


X=6.5 

s.d. 3.04 n=10 


X-7,6 

s.d. 4.48 n=ll 


X=7.4 

s.d. 4.67 n=11 










1 



ierIc 
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Table 17 



Analvsis Of Variance of the Immediate and Delayed 
^ th^Scrambled and Ordered Sequence Groups Experimenting 
OnrTwo. Sr Three Cycles Through the Program 



Source 



S.S. 



d.f. 



M.S. 



Between Ss 



1479.55 



57 



Cycles 

Sequence 

Sequence x Cycles 
Error Between 


76.14 

.14 

11.29 

1391.98 


2 

1 

2 

52 


38.07 

.14 

5.65 

26.77 


1.42 

<1.0 

<1.0 


Within ^s 


1715.00 


58 







Immediate - VS. 
Delayed Retention 


1311.21 


1 


1311.21 


180.17 


Cycles X Immediate 
vs. Delayed Retention 


11.16 


2 


5.58 


-Cl.O 


Sequence x Immediate 
vs. Delayed Retention 


1.69 


1 


1.69 


<1.0 


Sequence x Cycles x 
Retention 

Error Within 


12.50 

378.44 


2 

52 


6.25 

7.28 


<1.0 


Total 


3194.55 


115 







\gm 



p 



P 



5 i 

3 S. 

I i 



:.001 




The means 5 standard deviations^ and analysis of variance 
of the Student Reaction Inv entory measure of student attitude 
towards c omp u te r a s s i s t e d i Fi t r u c t i o n are shown in Tables 18 
and 19. The effects of the treatments on attitude towards CAI 
were statistically nonsignificant. 
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Table 18 






Means for Attitude Towards CAI for 
the Six Treatments Combinations 

Attitude Immediately Following Instruction 

Sequence 







Ordered 






Scrambled 




Number 


1 


X=78,0 
s.d. 9.38 


n*6 


s.d. 11.35 


5i«72.0 


n»9 


of 

Cycles 


2 


X=70.6 
s.d. 15.02 


n*8 


s.d. 7.39 


X=72.2 


n«12 




J 


X=77.8 
s.d. 9.79 


n*8 


s.d. 10.64 


j<*71.0 


n=12 



Attitude One-Month Following Instruction 

Sequenc'^ 



Number 

of 

Cycles 



Ordered Scrambled 



X=78,7 
s.d. 7.09 


X«74.2 

S.d. 7.33 n-9 


X=66.4 
s.d. 13.60 


X*71.7 

s.d. 16.91 n=12 


X-73.8 ' 

n*8 

s.d. 6.92 


X*66.7 

s.d. 12.62 




A 



Jl 



[ 
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Table 19 



Analysis of Variance of the Iminedi ate and Delayed 
Attitude Measures for the Scrambled and Ordered 
Sequence Groups Experiencing One, Two, or 
Three C: jles Through the Program 



!j 

[1 

] 

] 

] 

] 

D 

D 

D 



Source 


S.S. 


d.f. 


M.S. 


F 


P 


Between £s 


13288.70 


54 








Cycles 


390.02 


2 


195.01 


< 1.0 




Sequence 


189.87 


1 


189.87 


< 1.0 




Sequence x Cycles 


580.56 


2 


290.28 


1.17 


< .10 


Error Between 


12128.25 


49 


247.52 






Within ^s 


1955.50 


55 








Immediate vs. 
Delayed Attitude 


92.74 


1 


92.74 


2.70 


< .10 


Cycles X Immediate 
vs. Delayed Attitude 


144.89 


2 


72.44 


2.11 


< .10 


Sequence x Immediate 
vs . Del ayed Atti tude 


17.66 


1 


\7.66 


< 1.0 




Sequence x Cycles 
X Attitude 


19.24 


2 


9.62 


< 1.0 




Error Within 


1680.97 


49 


34.31 






Total 


15244.20 


109 
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Discussion and Conci usi onA 



Overall effects ojF course, sequencA 



Th« rPQiilts of the present series of investigations are in 

5 ip r. j 

"?Sa?Lr sequence ofln^truaiSrare not as large as has been 

previously suggested in the i°J'‘o?*'these*previ ous 

Setter sequencing. In general, the results 

'rt*!,?"rrndirafe thit cHtertori^t peK is non- 

nlfirantlv different for groups given scrambled versus 
Significantly V ereni ror yr uup y prpsent results do 

ordered programed instruction. Ho ^ error and instruc- 

not agreS with two previous studies the effects 

tional time measures were nepcrted), with regard t° instruc- 

of item scrambling on errors r|| (1962) compared 

tion and instructional time. Roe. .ase, and Roe 

a scrambled versus ordered I fnund nonsi qni fi cant 

^obability using college student -^fj^ucti onal 

differences on errors ,S?na aS iort^uctiona^pr^ in 

time. Levin and Baker (1963) using 

informal geometry with second grade children oo^^^ scores, 

nonsignificant effects for error “"Lus of thrpresent pilot 

significantly increase errors and ’"®*J“®r°ctional sequence did 

results indicated that the modern mathematics 

decrease the efficiency of instruction for tne^m^ 

?[iartcrambltrg“5ld n"ot effect the final ^Ip^l of criterion 

rcSiri-j j;|p3t r 

5s;'rsl;i:rs«r.n!r """ 

These results were frankly ® .^liJiPJ^’g^ej^gptilism'^of^so^^ 
Motivation for the expected to^obtein large and 
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the effects for the mathematics program were much smaller than 
had been expectedo For example, Table 4 shows that In Experiment 
I, the instructional time variable was significant at only the 
10 per cent level of statistical significance, and that the 
scrambled sequence ^s averaged only nine minutes longer to complete 
the two-hour program than the ordered sequence groupo 

There are several possible explanations for these results. 

The investigators believe that Experiment I avoided several 
ambiguities of interpretation which were present in earlier 
studieSo It was possible to conclude that the students employed 
in the study were almost totally naive with regard to the con- 
version of numbers to a nondecimal baseo The pretest performance 
of the Ss indicated that they had literally no idea of what was 
required for solution of the problems prior to instructiono This 
result contrasts with earlier reports In which naivete with regard 
to the subject matter wus either assumed, or when ^s achieved a 
high level of pretest performance pri or to instruction indicating 
a high degree of prior knowledge of the subject matter (see for 
example the recent study by Payne, Krathwohl , and Gordon, 1967, 
in which a group which did not receive Instruction obtained a 
relatively high mean score on a measure of achievement of elemen- 
tary concepts in psychological measurement) o If the S.s employed 
in an experiment already know a substantial amount of the subject 
matter to be taught, it is extremely difficult for the experiment 
to detect any treatment effects, let alone the effects of Instruc- 
tional sequencing. Experiment I minimized the effects of S^s * 
prior knowledge of the subject mattero With the exception of one 
or two S^s who achieved one or two points on the pretest, all 
S_s obtained pretest scores of zero on the ability to convert 
numbers from one number base to anothero 

Another problem which ha arisen in previous studies of 
sequencing concerns the effectiveness of the instructional 
program employed In the experimento If the "ordered" sequence 
program is only modestly effective in producing mastery of the 
subject matter, it is unreasonable to expect very large differ- 
ences between that program and a scrambled sequence version. In 
addition to the problem of the Ss prior knowledge of the material, 
the failure of the instructional program to produce mastery also 
decreases the sensitivity of the experiment to the treatment 
effect. Figure 4 on page 22 indicates that the number systems 
program employed in Experiment I produced a high degree of 
mastery. The reader should recall that the ^s whose criterion 
performance Is shown in Figure 4 all obtained scores of zero on 
the pretest of this ability. Thus, the program achieved its 
objectives for the large majority of the ^s. The lack of a sub- 
stantial treatment effect in Experiment l”“cannot be attributed to 
a failure of the instructional program to teach the intended 
material. The writers believe that the selection of naive S s , 
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ayid the effectiveness of the j^stnictl onal program maximized the 
sensitivity of the experiment to the possible effects of 
bled sequencing ((assuming of coyrse that such an effect exists tor 
the subject matter and popyl at. 1 on used|o 

What then could have accounted for the lack of a sequence 
main effect (aside from the error score) jn Experiment ^ 
possibility, ^»h1ch has created difficulties jin 
of previous findings. Is that the* a £ri£rT„ gudgmenc of 
tigators concerning the Inherent hierarchical 
modern mathematics program was Incorrect. 
did not have an inherent "order" 

one would not expect the sequence of instruction to be a crucial 
variable. In fact, this was precisely the rationale for Ex- 
periment I! which employed a topic seiiimgly devoid of a Jon- 
ceptua! hierarchy. The lack of a conceptual 

subject matters may account for the ^ absence of sequence eifects 
In the three previous experiments discussed above CRoe, 

IScI rL 1952- Levin and Baker, 1963; and Payne, Krathwohl, and 
Gordon ,* 19671 . Each of these studies failed to find statistical y 
sionificant differences iin the number of errors eommitted during 
instruction for groups given a scrambled as compared to an 
i?de?ed seSuenw It il difficult to believe that scramb ing the 
sequence of an inherently hieraifchieal subject matter would not 
increase the frequency of errors made during instructiono To 
re^SrHo an earlier sample. If a subject matter has the inherent 
concantual sequence A, B, C, and D, and one teaches B, C, 

or D first in the program it is literally impossible for a subject 
?o LU^problems correctly in these segments w thout the prereq- 
uisite training (assuming the hierarchical model is co.rect for 

given sub jict matter). Measures of program performance such 
as errors and^instructional time are the only independent 8™P’[’tal 
tests o?»e has for determining the validity of an assumed, a £riq rj _^ 
conceptual hierarchy. If within program P®»”f°tmance measures are 

not affected by altering the sequence^of J^^fj^lKrtnelcIpabll^ 



ordered subject matter, --es- .c. ^ + 

conclusion that the original hierarchical mooel °^J^he sequential 

characteristics of the subject matter was 

take the lack of significant differences on within P[9S‘8|J. 

measures in these edrlier studies as strongly suggestye that th^ 



01 



le IS 



were not In fact 



@gi ca 



ly" ordered, at least 



rt'irthni:re“orrhe"con;epi«i hiera;^hy^modei:-suggyted by 
Gagne' (1962) which implies that mastery of iLpis 

thi hierarchy is essential to the mastery of tasks at higher leyls 
in the hierarchy. Destroying the sequence hy ye hyry 

chical model for any given subject matter wo}*’** iL^Salid*for 
performance on higher level tasks assuming the model was valid for 

that subject matter. 
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The results of these previous studies may be contrasted with 
the results of Experiment lo The fact that the scrambled version 
of the modern mathematics program did produce a significant in- 
crease In within-program errors strongly suggests the presence of 
a facilitative sequence In the ordered version of the progratpo 
Performance on later program segments was facilitated by having 
had the necessary Instruction on earlier prerequisite segmentSc 
The skeptical reader is again referred to Figure 6 page 29« The 
dashed performance curve shows the performance of the scrambled 
sequence Ss on blocks of ten scrambled frameso The mean number of 
errors starts much higher than that for the ordered sequence 
group and does not drop below the ordered group until the last 
twenty frames of the programo The broken dash-dot performance 
curve shows the Increase in errors resulting from the scrambling 
of the original blocks of ten frames o The writers take these 
data to support their contention that the ordered version of the 
modern mathematics program employed in Experiment I did contain 
an ordered conceptual sequenceo We would conclude that the lack 
of a detrimental effect fer scrambled sequencing on the criterion 
test therefore, did not result from a lack of conceptual sequence 
In the subject matter program, and must be accounted for through 
some other explanationo 



Several explanations have been offered by researchers to 
account for the apparently small effects of scrambling the sequence 
of an Instructional program on student performanceo Koe, case, 
and Roe (1352) suggested that scrambling the sequence of instruc- 
tion increased student motivation to master the task, and conse- 
Quen+ly increased the strength of reinforcement when the student 
finally did achieve insight into the materiaU . 

state that: "Presenting items out of sequence possibly introduced 

a task oriented anxiety which was subsequently relieved in ® 
moment of revelation when a missing clue was discoveredo 
Case, an: Roe, 1962 , po 104) The only evidence in the P’^^jent 
studies bearing on this hypothesis were th^ responses f 
to the Student Reaction I nventory, in the pilot studyo Students in 
the scrambled seance condition reported being significantly 
more tense than students 1n the ordered program groupo fhese 
student self-reports did support the contention of Roe, Case, and 
Roe that item scrambling increases anxiety, however, 
creased student tension actually facilitates 
able Question which will require further researcho In fact, 
Contrary to the notion of Roe. Case, and Roe. previous research 
on the effects of anxiety on learning strongly suggests ‘•’f* high 
anxiety is detrimental to the learning of complex 

Ind measS^elent also luggests that high anxiety in the students 
can interfere with learning. 
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Payne, Krathwohl, and Gordon (1967) suggested another inter- 
pretation which relies heavily on the organizational powers of t 
learncTo These investigators state that the scrambled sequenc 

program, 

".ooWas viewed as a large step program, the individual 
sortinq out items that were previously contiguous or 
nearly so and recalling from memory the items that relate 
to a currently attended problem as appropnateo Incorrectly 
solved Items were reviewed in relation to the 
answer, to determine the correct principle, relation or 
facto In many instances this resulted in a discovery and 
inductive development of information and principle that 
permitted the students to learn the material involved at 
an earlier point (with fewer steps) than they . 

gone through in the logically sequenced prograiHo Thus the 

learner inductively built Almost 

he went through the program, m many instances using almost 

a discovery method of learning." (Payne, Krathwohl, and 

Gordon, 1967, pp. 131-132) 

The results of Experiment I appear to support this inter- 
pretation. 'Figure 6 on page 29 shows the 
(as measured by mean number of errors per block 
the scrambled and ordered sequence groups. The mean number of 
errors for the scrambled sequence group started 

higher than that for the ordered L-ckina 

hinifip?* throunh the first forty frames of the program. 
ufi^foSSn^necesslry for\he ®°’“t 1 on of the problems he^^^^ 
encountered during the early segments -he 

bled sequence Ss naturally had no trend in th^ 

rial and made more errors. However, the g . 

performance of the scrambled 9r°“P in 

towards a decrease in err 9 rs, until by se- 

the program they were making fewer errors than 
ouence arouD. In spite of the scrambled sequence and the «ni 
tiaiiv hioh^error rate, these Ss were apparently able to reor- ^ 
Sanize the material Ld eventullly achieved mastery /^®. 

?f the orogram. This interpretation is further supported by the 
f 1 nH-i nn^that When the scrambled sequence S^s were given the 

e^c^u^trrrd rn^lhrr"oSJi:"TSelf 

earlier material which was relevant to the solution of la 
problems . 
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The ability to reorganize scrambled material Is undoubtedly 
a function of the cognitive development of the learner^ Although 
It appears likely that college students are able to accomplish 
Such reorganization, the writers would be extremely reluctant to 
generalize such a conclusion to the problem of sequencing 
learning materials for young childreno It may be that sequencing 
is much more crucial in the education of young children who have 
not yet developed their own learning strategies* 

Although the writers do not view the present results as 
justification for ignoring course sequencing in the preparation 
of instructional materials- we do believe that the present results 
together with those of a number of previous investigations raise 
some embarassing questions for an approach to instruction which 
places a strong emphasis on optimal course sequencing, small step 
programs, minimal error rates, etc. Our results suggest that 
students are highly adaptable to a scrambled sequence of instruc- 
tion and are able to bring their own organization (and perhaps 
more meaningful organization for them) to bear on the subject 
matter. Although the effect of sequence does seem to depend on 
the nature of the subject matter (as Experiment II indicates), 
and on the level of cognitive development and abilities of the 
learners (as the interaction data In Experiment I suggests), we 
would venture the tentative coijclusion on the basis of the 
accumulated research to date, that instructional sequencing may 
be an overrated variable* Much research in the area of systematic 
instruction has focused on what may be relatively trivial manipu- 
lations of the instructional stimulus (e.g*, sequence, length of 
frame, size of step, overt versus covert response, feedback, 
etc.). Such manipulations of instructional stimulus variables 
may account for only a very small portion of the variance in 
student learning, compared to the amount of variance accounted 
for by the learning strategies and information processing 
strategies employed by students. A purely stimulus-oriented 
approach to the development of instructional materials may 
produce only very limited gains, if the most important componant 
in the instructional process is the student with all of his 
aptitudes, interests, and his ability to discover the concepts 
of a subject matter regardless of minor variations In the nature 
of the Instructional stimulus or its presentation sequence. The 
writers were surprised in the present study by the ability of 
college students to achieve mastery of the material In spite of 
what appeared to be a most serious distortion of the sequence of 
the instructional material. Studies of the information processing 
strategies employed by learners in dealing with different varia- 
tio*is in the instructional materials might provide more insights 
into how one might optimize the learninq experifince than compar- 
isons of arbitrary manipulations in the instructional stimulus 

per se. 
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If one examines the genesis of the principles of programed 
Instruction such as small-step, minimum error rates. o»;a®»'ed 
seouence. etc., one finds that most of these notions stem from 
research by Skinner and his colleagues on the learning of a very 
low leCel of animal life (®- 9 - .the pigeon) . This resear^^ led 
to a series of empirically based S-R laws which essentially 
ignorld the subject or " 0 “ variables. Perhaps with an 
which is as stimulus bound as the pigeon, one can afford 
the " 0 " variables, however, one runs great risks in 9®^® 
laws derived on pigeons to complex subject matter learning with 
hSmIn brings. As one prominant psychobiologist recently put it 
inSmmInting on the dangers of generalizing research findings 
from ni aeon to man. "The pigeon doesn't even have a cortex. 
Obviously the college students employed in the 5® 

a co?tei! and our rLults suggested that they were able to put 
their cortex to good use in making sense out of a morass of 
inciLctional inputs. Ignoring such learner variables in research 
oS instri^tion may lead to misleading conclusions the 

impona^crof cS^tain stimulus variables., ".ay hamper progress 

towards an individualized instruction which should take into 
a^coSnt tKe cai«i?ies of the learner for processing information. 

Interactions between student individual 
differenceT~and instructional . usnce 

cauarai nrovious i n v 6 s ti Q a t I oos have studied the problem 
of the interaction (letween student ‘‘’"||Jb^erse%enL 

providing no evidence for an ^.up ,^^ 5 + oro- 

^^"S'^e^rL)’^!^thr:ix^d1e?u;nci? 

determine which fraction would "®"t. In the consecut^ 

sequence, the student, assuming he had learned the nat^ 

sequence program, but did not correlate signirii.ai. j 
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performance on the consecutive sequence program’ Stolurow (1964) 
Interprets these respjslts as Indicating that® "ooothe best sequence 
did for the poorest ability group what the highest ability groups 
could do for themselves legardless of sequenceo" (Stolurow, 1964, 
Pc 351) In terms of the rationale of the present series of 
investigations, the high ability students were presumably able to 
reorganize the material on their own, whereas the low ability 
students were unable to reorganize on their own and had to rely 
upon the organization built into the Instructional programo It is 
important to note In considering Stolurow's results In relation to 
the results of Levin and Baker (1963), Payne, Krathwohl, and 
Gordon (1967), and the results of the present investigation, that 
Stolurow's students were educationally mentally handicapped ^So 
It Is entirely possible that the aptitude by sequencing interaction 
exists when one is dealing with students of extremely low ability, 
but that the effect 1s nonexistent when one 1s dealing with a 
more representative sample of ability, or with a more able group 
of college studentSo 

The results of the present series of Investigations, although 
they do not present an entirely consistent picture, do provide 
some evidence which Is supportive of the original hypothesis 
concerning the aptitude by sequencing interactlono The unexpected 
direction of the interactions obtained In the pilot study (ioeo, 
high aptitude ^s seemed to be most detrimentally affected by the 
scrambled sequence at borderline significance levels) were clearly 
not replicated In Experiment Ic In fact, the results of Experi- 
ment I conform to the results of Stolurov/ and support the notion 
that low aptitude Ss are most detrimentally affected by scram- 
bling the sequence"”of Instructiono The methodological improve- 
ments of Experiment I as compared to the problems encountered in 
the pilot study lead the writers to place more faith In the 
findings of Experiment lo 

The unexpected Interaction which appeared 1n the pilot 
experiment may have resulted from the extreme novelty of CAI and 
certain characteristics of the unrevised modern mathematics 
program which was employed In the pilot experimento When the 
pilot study was conducted, CAI was very new on the Penn State 
campus, and for that matter very new throughout the country o 
Few of the students employed in the pilot study had the slightest 
idea at that time as to what CAI was all abouto The subjective 
Impressions of the 1 nvestl gators while running the subjects for 
the pilot study, was that the students evidenced a high degree of 
ego involvement and a desire to do well at this highly novel 
method of instruction* Anxiety generated by the extreme novelty 
of computerized Instruction, and anxiety stimulated by the 
difficulty ®'icountered by the high aptitude Ss (in view of the 
usual expectat-l oins ©f success of such groupsj, may have resulted 
in the unusual decrements 1n the performance of the high ability 
Ss. In addition, the scrambled program employed in the pilot 
study seemed to produce much more anxiety in the S,s than the 
revised program employed in Experiment I* This anxiety probably 
resulted from the fact that the modern mathematics program 
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employed In the pilot experiment was much more of a continuous 
verbal program than the revised program employed in Experiment I. 
Scrambling the original program, in addition to scrambling the 
conceptual sequence, also scrambled the continuous verbal flow 
of the program which seemed to cause a great deal of frustration 
and anxiety in the subjectSo Subjects in the scrambled group in 
the pilot investigation took on the average 45 minutes longer to 
complete instruction than the S^s in the ordered sequence group® 

This time decrement may be compared with the much shorter 9 minute 
decrement experienced by the scrambled sequence group in Experiment 
lo It was not unusual while running subjects for the P]lot 
Investigation, to have a student bound out of the experimental 
room and indicate extreme frustration with his inability to pgure 
out what the program wanted® Our impressions of the frustration 
and anxiety of the Ss in the pilot experiment were born out by the 
self-report responses to the Student Reacti on I n v e n t o ry_ which 
Indicated a high level of tension Tn the scrambled sequence-h';gh 
aptitude groups® The effects of frustration which were observed 
during the pilot study, simply were not observed during Experiment 
I The revised modern mathematics program, unlike the earner 
unrevised program, consisted of a set of frames which essentially 
stood alone as separate problems in the conceptual sequence® The 
revised program contained a minimum of sequential deoendencies 
based on the written continuity of the program® The disruption of 
the sequence of a verbally continuous program may cause learning 
difficulties which are not encountered when a program consists .of 
frames which stand alone, each providing a separate problem to be 
solved® In addition, by the time Experiment I was being conducted, 
CAI had been on the Penn State campus for several years® It is 
highly likely that students from the subject population used had 
SGGn vidGO tapGS snd othGr QGrnonstrsti ons of CAI $ thG possi- 

bility of anxiety resulting from an extremely novel form of 
computerized instruction may have been reducedo The unusual 
Interactions between aptitude and sequencing which were obtained 
In the pilot study may have resulted from high anxiety generated 
from novelty of the experiment, and the extremely frustrating 
characteristics of the original unrevised program® 

The more reliable data of Experiment I, the improved instruc- r 
tional program, and the complete random assignment of Ss to 
treatments leads the writers to the tentative acceptance of the 
original directional prediction concerning the nature of tne 
aptitude by sequence Interaction. Although the two methods of 
analysis (correlational and the factorial analysis of variance) 
were not equivalent in statistical significance, these analyses 
produced results which were consistent in direction and statistically 
significant in the case of the factorial analysis of variance® 

This difference in the sensitivity of the two methods of analysis 
may have resulted from the greater power of the factoirial analysis 
of variance for detecting true interaction effects® We take 
the results of Experiment I as tentative support for the conclusion 
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that the effects of scrambling an instructional program containing 
^logicll concrptual hierarchy do in fact depend to some extent 
nn thp antitudes of the learnero The results suggested that the 
high aptitude Ss were able to reorganize a uP*'®®®gr* 

of subject matter to an extent J‘'®4ecte^ 

b5 item slrambling^ However, here again the largest and statis- 
tically significant effects were for the error and 
The effect^for criterion test performance was of smaller, statis 

tically nonsignificant magnitude 



The differences in probability values obtained SSoI!?" 

lational analysis of the aptitude by ®®q®®"®® ’ 

h« been to dichotomize high and low groups 5“®’ 

difference measure and run .a factor! al ana ysis The Juestion 

En.r.F’"F;rs“riiir.i52!"'5rSis^ 

{iKSiiiiisr:;;; p?p..« =«« 

instructional treatments. 



Subject m atter characteristics, and the 
e f feet’s of sc rambled sequence^ 



of Item scrambling might ^!Jo. , that the effects of scram- 

matters employed. It ®®®5'®,tJ.’ .Jonal program would depend on 
bling the sequence of ®"J®®^^®®|n®‘.Puential dependencies among 

tS'pSr p«ppi>«d 
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of a hierarchy of concepts (Gagne", 1962). and the hierarchical 
model had been validated for that Jerwoal^^^ 

definition affect within prog ram performance. If 
consTsteT“^of a relatlv^y unrelated set of facts , then one wouio 
not explct the sequence of instruction to be a critical variable, 
PrevioSs research with the number systems and anatomy 
emoloved in Experiments I and II suggested that these subjscL 
matters were widely separated on the 

hierarchy (number systems program) to a ThesrSwo^ 

a set of discrete facts (anatomy of the ear P^osram), These two 

subject matters were chosen for the investigations ®“®"*Pr 

to Lximize the difference P[®9'[®'"\®"4nd1ca^^^ 

rnncPDtual hierarchy inherent in the contento As inaicai:ea aDove, 

the invest! qators expected the scrambled sequence to have large 

delri:rnuref?eltsTn%tudent learning of the hlerarchial program. 

hut little or no effect on the learning of the set of relatively 

discrete facts. The expectations with regard to the anatomy 9^ . 

thl el^progra; were confirmed, however, the 

for the number systems program were much systems 

exoected. Although scrambling the sequence of the 

program did effect within program performance 

and instructional time), supporting our a 

number svstf^ms program did contain a conceptual hierarchy, 

cSmoletion of instfuction students had achieved mastery of the 

cSntlnt as measSreS by an immediate criterion test in spite of 

the scrambled sequence. 

The results of Experiment II with the anatomy program indi- 
cated almost consilteH^ly nonsignificant differences between^ 
scrambled and ordered sequence groups on errors, instruct. onal^ 
time, immediate retention, delayed retention, and attit^ 
rai ThP nnlv two clearly significant effects were linube xur 
cCcies'throuoh the program and retention over time reflecting the 
fai Sre of ?fcall oS tL delayed retention measure. These data 
tend to support the conclusion that item sequence is impor- 

tfnt fSr n?ogram of di: rete facts than for a program containing 

a conceptuel hierarchy. 

One Qualification on the findings of Experiment II 
the matterof the comparative length of t»e two programs e-nployed. 
Tn attribute the lack of differences in Experiment IT to a <acK 

If f conceptual hiera.chy in the anatomy P'^®^:;?:;* “in lenqih 
been desirable to have a program which }® ® 

in length) indicated that students could recall very few ot tne 
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j. ■f-av>mc aftpr onlv oHG cvcIg thTOugh thG prog^^^® ^ 

anatormcal tGrtns after ^ «- hi"ahlv analogous to a 

srrSr«roJus.'L2i.?:;; ?.K ’ . 5 - 

gi;°ciraUrfbulLirlo ^hrseruence^r/presentatio^ As the reader 
can see in S'""® 1° • Hill'll iKrJrSgJaraTas insufficient 

irSdu2e°Ka?rierrning as Indicated by the ;;el«tive|y high 
Trror rates during the first 'y^’®,*hrough^the Program.^ J^^ssary 

ro'cycie^stSdenU IhroSgh IJie^rogram a number of times in order 

to obtain an adequate 1®''®’ “„oSfdTave SecLsiraled 

cycles.would have ^e^n sufficient, Jhis^w^^ 

approximately SIX -tudent) in addition to approximately one 
of computer time per stuoenej, in a oossible to expect 

to two hours of criterion testing. It was not possine 

such large time 1°"'"’^ j°heen oossible to use the two-hour 
investigation. Even if it had been possioie lo uv 

version of the anatomy program it is not at all ^ sylteL and 
writers in what sense the 'length of the numoer^sy 

anatomy P'^99'^e?® .'''°“l^tS®iLrnlSa wi th ^program involving the 
a program Involving rote earn ng «;*h a progr 

learning of meaningful Principles produces ui^^ 

level of learning achieved. _ „ould have been approximately 

through the two-hour anatomy prg number sytems program, 

equal in length and "umber of frames t°^”e^numoe^ ^y^^^ 

however, the level of 1 ?h,<niinh the two-hour number systems 

that obtained with one cy?’® cont^ facilitated 

program in which t^e meaningful nature of the 

learning. The question is do you equate program g 

level of learning achieved through the Pr'>g|;®'"|- possible. 

was to attempt to equate ^^e progra ^ to tn fono„ing instruc- 

in terms of the l®v®l L chos^a sLrter anatomy program, 

tion In the ordered versions. computer time usage and 

(nGCGSSi tatGd in part oy tne aei nracticG cyclGS through 

student time), and varied the n P g program which 

the program from one to three within the time limitations 

produced an adequate level 1®®^I constant for the 

available. If we had held instruct amounts of learning 

two programs we ® suits would have been confounded 

in the two Pnognams. and the results^wo^ gchieved in the two 

with differences i" ^ Aai-iiioA that it was more important to 

?5;ri..“rr5Sl'.McJ ,ro«c,. 

!r,ni,r.;ir;s3»*5r..'-:i«5 u..-. .« 

learning in the anatomy program. 
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In SDite of the abo¥8 difficulties encountered in equating 
the length of Instruction for the two subject matters + 

systems and anatomy of the ear) the writers are fairly confident 
that the results obtained with the shortened version of the 
aniLmy Program would be predictive of resu ts obta ned with a 
iSnoerversiorof that program. Considerable experience w th 
thisoroa^ara suggests that the anatomy subject matter is almost 
toial?rdevSld“§l a conceptual hl!j;«^chy Althoug lengthening^ 
the program would have increased the number 
reach an adequate level of would 

alter the present results which indicated a lack of efrecc 
to scrambling the sequence of instructiono 

Comouter-assi sted i nstructi on £s_ a_ vehi cl,e 
for research on Tnstruction 

To the writers’ knowledge, the present project 
tho first attemots to conduct research on variables related 

Jotentially powerful *'®search tool for investig^ 
instructional process. I" we are nclined^to^^^ 

?h:%%“rr:^!e^l mjre obvlous^^ 

l"orfcrnrroriverrhe1;eseSlalifn’rf the instructional treatments. 

r"apiS %rv??lin“of"cArn is possible f®^t''!,,^^f„rtrSc?ional 
produce almost instantaneous variations the instructional 

tr^tment. For example, in Experiments I and 

decision logic of the i* hasic set of programed 

and response latency measures o 

Although there are many advantages to CAI 
Investigator just starting J" early^phases 

Of nis researCiio n jr . , _ _ e p — down oer^odo This 

tional system requires a ^JSct^aeneration of CAI systems, 

would be espaslally true examole the IBM 1500) have 

The second generation systems (for g^^er.ces of 

incorporated many improvements based on the exper 
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researchers with first generation systems. Somi of the problems 
which have been encountered during our own research with a p’^st 
generation CAI system have been interruption of Ss during the 
experimental Instruction resulting from machine failure, loss of 
student records in storage resulting from a machine failure, 
difficulties encountered in attempting to retrieve student 
performance data from computer storage for experimental analysis, 
and errors in compiles of experimental instructional programs 
Introducing systematic and unsys-^temati c variations into the 
instructional treatment. One great aid to CAI research would be 
Improved programs to retrieve student performance data in a form 
appri^priate for analysis by the investigator with a minimum of 
turn around time. Improvements of the type required to make CAI 
a more useful research tool will undoubtedly occur as more 
experience is gained in the CAI field. 



Summary 

The present series of investigations was undertaken to clarify 
the findings obtained in several previous studies which suggested 
that scrambling a “logical" sequence of instruction did not appear 
to have a significant detrimental effect on learning (see for 
example Roe, Case, and Roe, 1962; Levin and Baker, 1963; and Payne, 
Krathwohl, and Gordon, 1967). These results somewhat counter 
to the educational folklore concerning the importance of instruc- 
tional sequencing in subject matter learning, 

Investigators posited that the effects of scrambling the sequence 
of instruction probably depended to some extent 
of the subject matter being taught and individual differences 
among the learners. If a subject matter contained a s^equential 
hierarchy of goncepts (such as that suggested by Gagne, 1962} 
then one would expect scrambling the sequence to be detrimental 
to learning. If on the other hand, the subject matter ''onsisted 
of a relatively discrete set of facts, one would not expect the 
sequence of instruction to be such a critical variable. One 
pilot investigation, and tv^o complete experiments were undertaken 
to test the effects of scrambled versus ordered course sequencing 
with two subject matters. One subject matter in modern mathematics 
wars selected because it appeared to contain a hierarchy of con- 
cepts. The second subject matter on the anatomy of the ear was 
selected because it seemed to represent the other extreme con- 
sisting primarily of a set of relatively unrelated facts. 
expected that the scrambled instructional sequence wouid be highly 
detrimental to the learning of the modern mathematics program, but 
have little or no effect on learning in the anatomy program. In 
addition to the subject matter considerations, ^ ^ 

between the aptitude of the learners as measured by the Sc hoi asp c 
Aptitude Test (College student Ss were employed) and instructional 

sequence were examined to test the prep'pion ® thP^learnlna 

sequence of Instruction would be most detrimental to the learning 
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of low ability students, and that the high 

learn effectively by either method. The concnu-ions dr 

the three investigations were as follows: 

1) As prpdicted, the effects of scrambled „ 

and^’instrSItiSnirtime^in ^rmode emlticr^^^^ 

of instruction as measured by an J2!H®!h modern 

Increase in errors and modern 

mathematics program supported J**® ^“gptyg^'h^erarchy. An analysis 
mathematics program ® '“I^denU 

of the error data indicated that stuaencs in iiie s sequence 

to that of the ordered sequence groupo 

studentSo 




77 



Re ferences 



Florida State University studies computer's teaching potential 
Data Processing for Education s DeCo 1964s 3,5 14-15o 

Bitzer, D. L. and Easley, Jr., J. A. Plato: A computer-controlled 

teaching system. Paper presented DAVI Convention, Rochester, 
New York April 22, 1964. 

Boneau, C. A, The effects of violations of assumptions underlying 
the t«test. Psychol ogy Bui leti n , 1960, 5_79 49-64. 



Braunfeld, P. Go Problems and prospects of teaching with a computer, 
ii. of £d^ PSX:.. 1964, 55, 201-211 , 

Brown, E^obby R. An Instrument for the Measurement of Expressed 
Attitude Toward Computer Assisted Instruction, In Semi 
Annual Progress Report, Experimentation with Compute r-Aiil llleq. 
Instruction in T echnical Education . Project No, 5-85-074, 
f»repared b7 Harold E.' MTtzel et Vl, ,MDecember 31 , 1966 o 



Cronbach, L. 0. The two disciplines of scientific psychology. 
Amer . Psychologist , 1957, ^2, 671-684. 



Evans 



J. L. On investigation of teaching machine variables using 
learning programs in symbolic logic. Unpublished doctoral 
dissertation. University of Pittsburgh, 1960, 



Gagne, R, M. The acquisition of knowledge, Psychol. Rev,, 1962, 
69, 355-365. 



Glaser, R. Principles of programming. In J. 
Programmed L earning . Ann Arbor, Mich^ 

Re s e a r c h ^n^Wuma n Behavior, 1961s 7-20. 



P. Lyrought (Ed.), 
Foundation for 



HamiUon, Nancy R. Effects of logical versus random sequencing 
items in an autoinstructional P»'og»'a'",!;" 5 er two conditions 
overt response. Educ . Psychol . 1964* 5o.> ^58-266 . 



of 

of 



Levin, G. R, & Baker, B. 
ti onal program . 



Lo Item Scrambling in a self-i nstruc- 
EduCo Psychol . , 1963, 5^, 138-143. 



Lumsdaine, A. A. Instruments and media of 

Gage (Ed.l, Handbook of Resear ch on Teaching . Chicago 

Rand-McNal ly , 1963, 583-F8TI 



N. Lo 
, 111 0 : 



Mager, R. F. and Clark, C. Explorations in student-controlled 
instruction. Psy . Reports , 1963, 1 3 , 71-7o. 



78 



Maher, Anna. Computer-based instruction (CBI): Introduction to 

the IBM projecto IBM. Research Reporto RClij4, j964o 

Mitzel, He Eo The development and presentation of four college 
courses by computer teleprocessingo Interim Report, United 
States Office of Health, Education, and Welfare, Project 
#0E-4-16-010o Revised April, 1966r 

Mitzel, Ho Eo and Wodtke, Ko Ho The development of four different 
college courses for presentation by computer 
Interim Report, United States Office of Education, Department 
of Health, Education, and Welfare, Project NOo OE-4-16-010, 
June, 1965. 

Norton. D. W. An empirical investigation of some effects of non- 
normality and heterogeneity on the F-distnbution. 

Unpublished doctoral dissertation. State University of Iowa, 

1952. 

Osaood C. E.. Suci , G. J.. and Tannenbaum. P. H. The Measurement 
of Meanin g. Urbana: University of Illinois Press, 1957. 

Payne, David A.. Krathwohl. David R. , and Gordon, '3°'’;’” JJ)® 

^ of sequence on programmed instruction. Amer . EducationaX 
Research Journal , 1967, £, 125-132o 

Roe. K. V. Case. H. W.. and Roe, A. Scrambled versus 

sequence in autolnstructional programSo Edu£o Psyc h, q 2 

1962, 53, 101-104o 

Skinner, Bo Fo Teaching machineSo Scler^, 1958, 11£, 969-977o 

Stolurow. L. W. Social Impact of Programmed 

and Abilities Revisited In J. P. DeCecco ' 

Technology, New York: Holt, Rinehart, and Winston. 1964, 

348-355. 

Stolurow L. M. and Davis, D. J. Teaching machines ®°'"P®*®*‘' 
based systems. Urbana. Illinois: J" 

Training Research Laboratory. Techn^ 1" 0^^’®® 

of Naval Research, NONR 3985 (04) , 19537 

Suppes. P. Modern learning theory and the elementary school 
curriculum^ Amer o Ed o Rej^ o ^ 1964, 7 o 

Uttal. W. R. My teacher has three arms 
IBM Research Report RC-7^ , jSo^o 



O J) 



0 Q 0 

o o o 



Yorktown Heights, N. Y 



o 




79 



Wodtke, Kenneth H. Scrambled versus ordered course sequencing 
In computer-assisted Instruction: Experiment 1 In Semi- 

Annual Progress Report, Experimentation with Computer- 
Assisted Instruction In techni cal Education . Project 
No» 5-65-074, PrepareBTby Harold E. Mftzel ejt aj[. , December 
31, 1965. 

Wodtke, Kenneth K. On the assessment of retention effects In 

educational experiments. Journal of Experimental Education 
Summer 1967, 35, 28-36. 



APPENDIX A 

AJ Sample of the Original Number Conversion Program 
A. 2 Sample of the Revised Number Conversion Program 



o 



83 



Appendix A.1 



Look for the highest power of eight represented in 724 (ten). 
What is it? 

Eight to the 3rd power is the highest power of 8 in 724 (ten). 
Type eight to the third power. 

light ?o*^the 3rd power is the highest power of 8 in 724 (ten). 
Type eight to the third power. 
eight to the thivd power 
Correct. 



What is the numerical value of eight to the third power? 
612 

512 Is correct. 



We subtract 512 from 724, resulting in 212. What power of eight 
do we look far in this number? 

We look for eight to the second power. Type eight to the second 
power o 

8 to the eeoond power 
Correct. 



How many groups of eight to the second power are represented 
in 212? 

1 

There are 3 g^roups of 8 squared in 212. Type 3. 

3 

Correct. 



We subtract 3x8 squared, or 192, from 212. 

How many groups of 8 to the first power does the result, 2Q, 

contain? 



2 

Correct. 



Subtract j the remainder is 4. Therefore, putting all of this 
work together, what is 724 (ten) in base eight? 

3324 , . . 

724 (ten) = 1324 (eight). Type 1324 (eight). 

1324 

Correct - 724 (ten) « 13?4 (eight). 
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Look again »t the ehart in display 5-E. What is the value in 
bss@ ten of the 211 (eight) shown? 

IS? 

Correct 



What is the value of 47 (eight) in base ten? 
39 

Correct 



What is the value of 526 (eight) in base ten? 



Think carefully--it's 5 times 8 squared (the base squared) plus 
2 timers (the\ase) plus 6 ones. Now-whafs the value of 
526 (eight) in base ten? 



342 

Correct 



What is the value of 323 (eight) in base ten? 

Let^s look again. We have 3 groups 

groups of 8 or 16 plus 3 oneSc Total 192 +16+3. 

(eight) in base ten? 

s s ^ 

Let's look again. We have 3 groups 

groups of 8 or 16 plus 3 ones. Total 192 +16+3. 

(eight) in base ten? 

Let's look again. We have 3 ,®f 

Groups of 8 or 16 plus 3 onoso Totsil 192 <» 

(eight) in base ten? 

211 

Correct 



192 plus 2 
What is 323 



192 plus 2 
What is 323 



192 plus 2 
What is 323 



IS 



the value of 2372 (eight) in base ten? 



What 

?heck vour work. 2372 (eight) means 2 groups of 8 to the third 

SSwer plus 3 groups of 8 to the s®®®|;^.?®“r 64 

fi DOWSr dIUS 2 QHGS* Thftt S 2 X Difc p I U5 

plus 7x8 plus 2. When you add these together, what is the 
answer? 



1274 

RightI 



5-30 

Let's check back and review a bit. What number in base eight 
means 5 of the base and 3 ones? 

53 

Correct - 53 (eight) 
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What number follows 37 In base eigh.t^? 

38 

Remember the numeral 8 is not used in base eight. Try again. 
40 

Correct - 40 (eight) 



In 652 (eight), which digit shows the number of groups of 8 
squared, or eight to the second power. 



Remember the place value in base eight. Consider 652 (eight). 

From right to left, 2 is in the ones .Le 

is in the 8 to the first power place. What digit is, tnererore, 

in the 8 squared place? 
dRight! 



What is a logical first step in changing 95 (ten) to base eight? 

a^^^Look for*the largest power of 8 contained in 95 (ten). 

b. Divide by 8. 

c. Write ”5“ in the units column, 

d. Change the 9 since there can be no 9 in base eight. 



This method, involving repeated division by 8, can be meaningful 
for ascertaining the powers of 8, 



Actually, either a or b above were accepted as correct answers, 
"a" is i^ accord with the method taught in this program. For a 
more detailed explanation of the s*'®J*-put method in b, turn 
to display 5-F. If you decide to skip display 5-F, type go 
Otherwise follow the directions on the display* 



on 
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Appendix A. 2 



BEFORE BEGINNING THIS LESSON, THE INTMDUCTION^I^^^ 

BOOK OP EXHIBITS GIVEN TO YOU. WHEN YOU ARE READY TO i’KUOE.c.u, 

PRESS EOB. 

1 FOR THE NEXT FEW ITEMS , YOU WILL BE GIVEN A SM^ PROBI^ , 
AND A SAMPLE SOLUTION . BELOW THIS SAMPLE SOLDTIStT Ig-J 
PROBLEM WHICH YOU AbE TO SOLVE. 



SAMPLE PROBLEM; 2(10^) « ( SEE SOLUTION BELOW ) 

SAMPLE SOLUTION; 2(10^) = 2(10 X 10) = 2(100) - 200 

PROBLEM; 

NOTE; 



5(10^) = 5(10 X 10) = 5(100) = JL 



5(( 

NO. 



2 . 



WHENEVER YOU SEE A BLANK SPACE LIKE 1— _ 

YOU ARE TO TYPE THE ANSWER THAT 
IpACE; AND THEN PRESS EOB TO ENTER THE ANSWER 
INTO THE COMPUTER. DO THAT NOW. TYPE THE 
ANSWER, AND THEN EOB. 



THE CORRECT SOLUTION IS; 

5(100) = 500. 5(100) IS THE SAME AS 5 X 100. 

YOU SHOULD HAVE TYPED; 500. 

SAMPLE PROBLEM; 3(10^) “ ( SEE SOLUTION BELOW ) 

SAMPLE SOLUTION: SUO^) = 3(10 X 10 X 10) = 3(1000) = 3000 

PROBLEM: MlO^) = 4(10 X 10 X 10) = 4(1000) = _L_ 

pwimrimKR EVERY TIME YOU SEE A BLANK LIKE THE ONE 
?6u TSlrepi THE ANSWER THAT BELONGS IN 

THE SPACE AND THEN PRESS EOB. 



4000 

VERY 


GOOD. 






3. 


SAMPLE PROBLEM; 


4(10^) + 


3(10^) 




SAMPLE SOLUTION; 


4(10^) + 
30 = 430 


3(10^) 




PROBLEM; 


6(10^) + 
600 4- 20 


2(10^) 
+ _1_ 



620 

CORRECT 
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M SAMPLE PROBLEM: 2(10^) + l(lO^) + 2(10 ) = (SEE SOLUTIQ ^ 

BELOW ) 

SAMPLE SOLUTION: 2(10^) + I(IO^) + 2(10°) = 200 + 10 + 2 = 

212 

NOTE: 10° ALWAYS EQUALS 1; HENCE, 2(10°) * 2(1) 

PROBLEM: I(IO^) + 2(10^) + 3(10°) = — ? 



DO YOUR COMPUTATIONS WITH PENCIL AND PAPER. THEN 
TYPE THE ANSWER ONLY. 



223 

CORRECT o 

5. IN AN EXPRESSION StJOH AS THIS, 3(lo'‘). ME 3 IS CAL^D A 
pnFTTFTCIENT * THE 10 IS CALLED A BASEj AND THE^JJ 
EXPOTENT ” raWILL BE USING THESE TERMS IN THIS LESSON; 

tSpS^. you will need to learn them, for practice, type 

THE NAME OP THE 3, 10> AND THE 4 BELOW. 

COEFFICIENT^ BASE^ EXPONENT 
CORRECT 

6 IN THIS PROBLEM, 4(13^) > CALLED THE • 

"13” IS THE . "5” IS THE - 

COEFFICIENT j BASE» EXPONENT 
CORRECT. 



7. 



the digits we see in a Sp^I^^iSIpkcSs.) 



SAMPLE PROBLEM: 
SAMPLE SOLUTION: 
PROBLEM: 



EXPAND 49 

49 = 4(10^) + 9(10°) 

81 = 8(10^) + ? (lOP) 

YOU WILL NEED TO PROVIDE ‘IHE MISSING 
COEFFICIENT ONLY 



1 

CORRECT 



8. SAMPLE PROBLEM: EXPAND 321 

PAY PARTICULAR ATTENTION TO WHAT HAPPENS TO THE COEFFICIENTS 
372 ^D 1 IN THE SAMPLE SOLUTION. 
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SAMPLE SOLUTION: 321 = 3(10^) + 2(10^) + I(IO^) 

PROBLEM; 233 = ? (10^) + ? (10^) + ? (10^) 

YOU NEED TO PROVIDE THE MISSING COEFFICIENT. 



CORRECT 

9. WE HAVE SAID THAT THE DIGITS IN A HUMBER ARE COEFFICIENTS, 
AND THAT THE BASES AND EXPONENTS ARE UNDERSTOOD. IN THE 
DECIMAL SYSTEM IT IS UNDERSTOOD THAT THE BASE IS ALWAYS 

_? . 

10 

CORRECT 



10 . 



SAMPLE PROBLEM: 
SAMPLE SOLUTION: 



PROBLEM: 



EXPAND 320 

320 = 3(10^) + 2(10^) + 0(10°) 

THE NUMBERS 3, 2, and 0 = COEFFICIENTS 
THE NUMBERS 10, 10, and 10 = BASE 

912 = 9(10^) + I(IO^) + ? ( ? ) 



YOU NEED TO PROVIDE THE MISSING COEFFICIENT AND THE MISSING 
BASE., MAKE SURE TO SEPARATE EACH ANSWER WlTH THE SPACE BAR. 

CORRECT. 
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Appendix Bol 



CONO 

Date_ 

1 . 

2o 

3. 

4o 

5o 

6 . 

7. 

8 . 
9„ 
10 . 
11 . 
12 „ 
13. 



(PART IB) ■* ^ ^ Nme 

CAI Student Number 



What is 724 (ten) in base eight? 



What is 2372 (eight) in base ten? 



What is 321 (five) in base ten? 



What is 79 (ten) in base five? 



What is 23 (eleven) in base ten? 



What is 36 (nine) in base ten? 



What is 269 (ten) in base five? 



What is 3 (ten) in base two? 



What is 10 (ten) in base two? 



What is 22 (ten) in base two? 



What does 364 (seven) equal in base 10? 



What does 324 (eight) equal in base six? 



What does 115 (fifteen) equal in base ten 



14, What does 2 (six) equal in base seven? 
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15. 


What 


does 100 (four) equal i 


in base two? 


16. 


What 


does 225 (seven) 


equal 


in base nine? 

» 


17. 


How 


is one of the base always Written regardless of the value of the base? 


18. 


How 


is one of the base squared always written? 


19. 


How 


is one of the base cubed always written? 


20. 


■ rr 1 

What is the meaning of the 


6 in 632 (base nine)? 




a. 


6 ( 9 ’) 








b. 


6 ^( 9 '') 








c. 


6(9^) 








d. 


6(9^) 








e. 


6^(9^ 






21. 


What is the meaning 


of the 


5 in 15,342 (base six)? 




a. 


5(6^ 








b. 


5^(6^ 








c. 


5^(6^ 








d. 


5(6"^) 








e. 


5(6^) 






22. 


What is the expanded form 


of 3726 (base nine)? 




a. 


3(9^) + 7(9^) + 


2 ( 9 ^) 


+ 6(9^ 




b. 


9(3^) + 9(7^) + 


9(2^ 


+ 6 




c. 


3(10^) + 7(10^) 


+ 2(1ob + 6(1) 




d. 


3(9^) + 7(9^) + 


2(9^ 


+ 6(9°) 




e. 


10(3^) + 10(7^) 


+ 10(2^) +10(9°) 
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Appendix B„2 

POSTTEST (FORM T) 



NAME 



DATE 



Most people think that a number system must necessarily employ numerals; however, 
if the basic principles of number systems are understood any symbols may be used. 
For example, the hypothetical "fum society" uses the following number names with 

the base equal to fum: 







X 


X 


X X 


none 


X 


X 


X X 


X X 


fizzle 


fe 


fi 


fo 


fum 



The base squared = gum 
base cubed = hum 
base to the fourth = jum. 



1. Write the number in fum notation which is equivalent to the numeral 1233 
(in base four). Use commas to separate the place value positions. 



2. Write the numeral 1312 (base four) in fum notation: 




3c Show how we would count twenty objects in the fum system. 
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4, 



Add the following numbers in the fum system. Do not 
add the two nus*ers directly. Show all your work. 



convert to base ten. 



but 



fe-gum, fi-fum, fo 
+ I fo-fuiiie fl ' 



a. What value (in fum notation) is carried in column one 

b. What value is carried in column two? 



5. Try adding: 

fo-gum, fe-fum, fe 
+ f^■.guma £i~fume _fq 

a. Carried over from column one (in fum notation) = _ 

b. Carried over from column two (in fum notation) = _ 

c. Carried over from column three (in fum notation) - 



6. Let’s try adding some numerical values in the base eight 

472 (eight) 

+ -6 5 4 (eight). 



a. Value catried from column one =» — 

b. Value carried from column two * __ 

ce Value carried from column three « 




i 






7o Subtract 



325 (base eight) 
2 7 6 (base eight) 



a. Value borrowed in column one 

b. Value borrowed in column two 

8. Add the following numbers in base two o 

110 1 (two) 

+ 1011 (two) 



a. Carried from column one 

b. Carried from column two __ 
Co Carried from colu a three 



100 1 (two) 
+ 111 (two) 



a. Carried from column one _ 

b. Carried from column two __ 
Co Carried from column three 



8. (continued) 



10101 (two) 
+ 1011 (two) 



a. Carried from column one ^ 
b* Carried from column two ^ 
c. Carried from column three 
d» Carried from column four _ 

1111 (two) 
- 101 (two) 



a. Borrowed from column one 

b. Borrowed from column two 

c. Borrowed from column three 

1001 (t^o) 
111 (two) 



a. Borrowed from column one 

b. Borrowed from column two 

c. Borrowed from column three 



8 



(continued) 



1011 (two) 
101 (two) 



ao Borrowed from column one 

b. Borrowed from column two 

c. Borrowed from column three ___ 

9. Add the following base five numbers: 



3 4 2 
+ 114 



2 4 0 1 
+ 13 4 2 



carried in column one 

carried in column two 

carried in column three 



carried in column one 
carried in column two 
carried in column three 
carried in column four 



3 2 4 3 
+ 4 3 3 2 



carried in column one 

carried in column two __ 
carried in column three 
carried in column four 



o 

ERIC 



100 



10. Count to fifteen In the binexy eyetem. 



10/66 - eono 
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STUDENT ATTITUDE TOWARD COMPUTER ASSISTED INSTRUCTION- 

This is not a test of information; therefore, there is no one "right" answer to a 
question We are interested in your opinion on each of the statements below o Your 
opinions will be strictly confidential » Do not hesitate to put down exactly how you 
feel about each item^ 

NAME DATE___ 



CIRCLE THE RESPONSE THAT MOST NEARLY REPRESENTS YOUR REACTION TO EACH OF THE STATEMENTS 
BELOW 0 



U The method by which I was told whether I had given a right or wrong answer became 
montonouSo 



2 . 



Strongly Disagree Uncertain Agree 

Disagree 



Strongly 

Agree 



The material presented to me by Computer Assisted Instruction caused me to feel that 
no one really cared whether I learned or not„ 

Strongly Disagree Uncertain Agree Strongly 

Disagree 



While taking Computer Assisted Instruction I felt challenged to do my best worko 

Strongly Disagree Uncertain Agree Strongly 

Disagree 

4o While taking Computer Assisted Instruction I felt isolated and alone» 

/\n the Most of Some of Only Never 

time the time the time occasionally 



5„ While taking Computer Assisted Instruction I felt as if someone were engaged in 
conversation with mOo 



All the Most of 

time the time 



Some of Only Never 

the time occasionally 



6o As a result of having studied some material by Computer Assisted Instruction, I am 
interested in t^rying to find out more about the subject mattero 

Strongly Disagree Uncertain Agree Strongly 

Disagree 
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7, 



I was more involved in running the machine than in understanding the material 



All the 
time 



Most of 
the time 



Some of 
the time 



Only 

occasionally 



Never . . 



8. Computer Assisted Instruction makes the learning too mechanical. 

Strongly Disagree Uncertain Agree 

Disagree 

9. I felt as if I had a private tutor while on Computer Assisted Instruction. 

Strongly Disagree Uncertain Agree 

Di sag roe 

10. I found it difficult to concentrate on the course material because of the hardware. 



Strongly 

Agree 



Strongly 

Agree 



All the 
time 



Most of 
the time 



Some of 
the time 



Only 

occasionally 



Never 



11. The Computer Assisted Instruction situation made me feel quite tense. 

Strongly Disagree Uncertain Agree 

Disagree! 

12. Computer Assisted Instruction is an inefficient use of the student's time. 

Strongly Disagree Uncertain Agree 

Disagree 



Strongly 

Agree 



Strongly 

Agree 






Very 

Favorable 



Favorable 



Indifferent 



Unfavorable Very 

unfavorable 






Very 

Favorable 



Favorable 



Indifferent 



Unfavorable Very 

unfavorable 



15. While on Computer Assisted Instruction I encountered mechanical malfunctions. 

very often Often Occasionally Seldom Very Seldom 

16. I felt frustrated by the Computer Assisted Instruction situation. 

Disagree Uncertain Agree 



Strongly 

Disagree 



Strongly 

Agree 
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The Computer .Assisted Instruction approach is inflexiblOo 

Strongly Disagree Uncertain Agree 

Disagree 



Strongly 

Agree 



18. Even othewise Interesting material would be boring when presented by Computer 
Assisted Instructiono 



Strongly 

Disagree 



Disagree 



Uncertain 



Agree 



Strongly 

Agree 



19. In view of the effort I put into it. I was satisfied with what I learned while taking 
Computer Assisted Instructions 



Strongly 

Disagree 



Disagree 



Uncertain 



Agree 



Strongly 

Agree 



20 , 



In view of the amount I learned,.! would say Computer Assisted Instruction is 
superior to traditional instructions 



Strongly 

Disagree 



Disagree 



Uncertain 



Agree 



Strongly 

Agree 



?1 With a course such as I took by Computer Assisted Instruction. I would prefer 
Computer Assisted Instruction to traditional instruction. 



Strongly 

Disagree 



Disagree 



Uncertai n 



Agree 



Strongly 

Agree 



22, I am not in favor of Computer Assisted Instruction because it is just another step 
toward de-personal i zed instructiono 



Strongly 

Disagree 



Disagree 



Uncertain 



Agree 



Strongly 

Agree 



iU 
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a-lo Following the normal pathway of sound, type the names of 
the three main divisions of the earo 
outeT Bar, inneT ear^ and middle ew:s> 

Good* Another name for outer ear Is external ear* 



a-2o What Is the name for that part of the external ear most 

visible and on the outside of the head* 

pinna 

Correct* The terms pinna and auricle are synonymous* 

a-3o What small structure of the ear divides the external ear 
from the middle ear* You need a speculum to see 1t* 
ear drum 

Correct* However a more technical term we shall be using is 
tympanic membrane* 

a-4* What passageway connects the pinna or auricle with the 
tympanic membrane* 

auditory oanat 

This 1s one of several correct answers, however, the preferred 
term Is external auditory meatus* 

a-5* What Is the name of the substance secreted by the glands 

along the external auditory meatus? 

oerutin 

Cerumen Is correct* 



We have briefly glanced at the anatomy of the 
let's consider something about Its function* 
following multiple choice questions by typing 
the correct choice* 



outer ear* Now 
Answer the 
the letter of 



a-6* The function of the pinna or auricle In human beings can 
be described as: 

a* Sound concentration 
b* Support for women's earrings 
c* Sound directing 
d* Basically useless 
d. 

Type the lower case letter of the correct choice* 
d 

Anyone of the four alternatives has some truth In It* However, 
look at the fifth line from the top of page 18 In Newby for an 
important consideration In respect to this question* 



a-7. On humans the external auditory meatus has which of the 
following main functions for hearing. 

’ a. A receptacle for cerumen 

b. Aerate the tympanic . . . ... the t m. 

c. Equalization of ?Srthl^ 

d. Allows sound to arrive at the t.m. for furtne. 

hearing functions 

Though this, in fact, may occur, it is "o* t**® primary function 
of the external auditory meatus. Try again. 

Though this, in fact, may occur, it is function 

of the external auditory meatus » Try againo 

Though this, in fact, may occur, it is not the primary function 
of the external auditory meatus. Try again. 
d 

Correct 

a-8. What does one normally find along the external auditory 

meatus other than cerumen? 

hair 

Correct. 

a-9. What Is the name of part 0 of 
A helical spring is a coiled spring. The part o 

Feel the outer and upper edge y?“'^„®“ry®Ji;a^e°2-rand 
around. Find the comparable part in Newby Fi;,ure 

type its name. 

Good! Notice that the helix extends a*'?''"'' « 
ear--hence there are four letters 0 on the model. Write 
"helix" on the line next to 0 on your handout. 

a-10. Identify part M on your handout, «s”*there!*'*You^may 

not well developed on the model* but the ” rha*»ip« 

T,p. or p.rt «. 

pfrtafthe question. The answer is evident in the question. 
The^answer is Darwin's tubercle. Type it. 

Danoin*8 tubercle na*»w^n'c tubercle. Write it on the line 

The correct answer is Darwin s tuoercie. 

next to M on your handout. 



107 



a 11 Follow vour own hoi lx up 9 ftf’Ound 9 snd to 

I't lnachesVyour head. This is part E on your handout 

What do we call 1t? 

? 

Incorrecto Keep trylngo 
helix euvl 

Type: crest of helix 

CorrL?^ ^Thif anatomical part has at least three names, 
of helix, limb of helix, or crus of helix. Label part E 

your handouto 



unti 1 



crest 

on 
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C.l Posttest for the Anatomy of the Ear Program 



THE RIGHT HUMAN PINNA 




CAI SPA 430 
10/9/65 
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Appendix C, 1 


Date 


Name: 



THE EAR 



1. 


What is another technical anatomical name for the pinna? 


2, 


What is another name for the external ear canal? 
(three words) 


3. 


What is the technical name for earwax? 


4, 


In humans, the external auditory meatus has which of 
the following main functions for hearing? 




a, a receptacle for cerumen 

bo aerate the tympanic membrane 

c. equalization of pressure on both sides 
, of the tom, 

do allows sound to arrive at the t.m, for 
further hearing function 



♦Notes Ss* task was to fill in the blanks on this form and to label the 
picture of the ear on the next page. 



